Sep 09 06:49:13.499261 ip-10-0-128-206 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 09 06:49:13.499275 ip-10-0-128-206 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 09 06:49:13.499285 ip-10-0-128-206 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 09 06:49:13.499516 ip-10-0-128-206 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 09 06:49:23.689232 ip-10-0-128-206 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 09 06:49:23.689247 ip-10-0-128-206 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 6da8ae3eb3914bc2aeae5dbe4c727c83 --
Sep 09 06:51:07.500900 ip-10-0-128-206 systemd[1]: Starting Kubernetes Kubelet...
Sep 09 06:51:07.948702 ip-10-0-128-206 kubenswrapper[2563]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 06:51:07.948702 ip-10-0-128-206 kubenswrapper[2563]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 09 06:51:07.948702 ip-10-0-128-206 kubenswrapper[2563]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 06:51:07.948702 ip-10-0-128-206 kubenswrapper[2563]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 09 06:51:07.948702 ip-10-0-128-206 kubenswrapper[2563]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 09 06:51:07.951555 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.951467    2563 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 09 06:51:07.954783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954768    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 06:51:07.954783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954783    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954787    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954792    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954795    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954798    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954800    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954808    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954811    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954813    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954816    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954818    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954821    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954824    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954826    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954829    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954831    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954834    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954836    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954839    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954841    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 06:51:07.954846 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954844    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954846    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954849    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954852    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954855    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954858    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954861    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954864    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954866    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954869    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954871    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954873    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954876    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954878    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954881    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954884    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954886    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954888    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954891    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 06:51:07.955329 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954894    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954897    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954899    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954901    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954904    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954907    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954910    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954913    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954915    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954920    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954924    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954927    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954930    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954932    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954935    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954938    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954942    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954946    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954949    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 06:51:07.956083 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954952    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954956    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954958    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954961    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954963    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954966    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954970    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954973    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954976    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954980    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954982    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954985    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954987    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954990    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954993    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954996    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.954998    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955000    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955003    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955006    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 06:51:07.956669 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955008    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955011    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955013    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955016    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955018    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955021    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955023    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955026    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955029    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955032    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955035    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955037    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955040    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955043    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955045    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955048    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955050    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955053    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955055    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.955058    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 06:51:07.957191 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957150    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957163    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957168    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957171    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957174    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957177    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957180    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957184    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957186    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957189    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957192    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957196    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957201    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957204    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957207    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957210    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957212    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957215    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957218    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 06:51:07.957695 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957220    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957223    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957226    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957229    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957231    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957234    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957237    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957239    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957242    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957245    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957248    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957250    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957253    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957256    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957259    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957263    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957265    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957268    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957270    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957273    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 06:51:07.958178 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957276    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957278    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957281    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957283    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957286    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957289    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957292    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957295    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957297    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957300    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957303    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957307    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957310    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957313    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957315    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957318    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957322    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957326    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957329    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 06:51:07.958687 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957332    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957335    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957338    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957340    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957343    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957346    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957348    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957352    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957356    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957359    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957362    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957364    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957367    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957369    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957372    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957374    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957377    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957379    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957382    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957384    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957387    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 06:51:07.959140 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957390    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957393    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957396    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957399    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957402    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957404    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957407    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957410    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957412    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957415    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957417    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957420    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957423    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957426    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957429    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957431    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957434    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957437    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957439    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 06:51:07.959677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957443    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957862    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957868    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957872    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957875    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957878    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957882    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957885    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957888    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957890    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957893    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957896    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957898    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957901    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957903    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957906    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957910    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957913    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957916    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957918    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 06:51:07.960155 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957922    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957926    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957929    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957932    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957935    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957938    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957941    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957943    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957946    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957950    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957953    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957956    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957959    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957962    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957964    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957967    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957969    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957972    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957974    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 06:51:07.960650 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957976    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957979    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957982    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957984    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957987    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957989    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957992    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957994    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.957998    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958002    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958005    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958009    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958011    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958014    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958017    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958020    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958022    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958025    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958028    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 06:51:07.961110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958030    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958032    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958035    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958037    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958040    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958042    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958045    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958048    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958050    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958053    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958055    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958058    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958061    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958063    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958066    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958068    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958071    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958073    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958076    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958078    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 06:51:07.961599 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958081    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958083    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958085    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958088    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958090    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958093    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958096    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958098    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958103    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958106    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958108    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958111    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958113    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958116    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958119    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958121    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958123    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958126    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958128    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958131    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 06:51:07.962117 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958133    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958136    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958214    2563 flags.go:64] FLAG: --address="0.0.0.0"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958250    2563 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958257    2563 flags.go:64] FLAG: --anonymous-auth="true"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958262    2563 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958266    2563 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958269    2563 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958274    2563 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958281    2563 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958285    2563 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958288    2563 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958291    2563 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958295    2563 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958298    2563 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958301    2563 flags.go:64] FLAG: --cgroup-root=""
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958304    2563 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958307    2563 flags.go:64] FLAG: --client-ca-file=""
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958311    2563 flags.go:64] FLAG: --cloud-provider="external"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958314    2563 flags.go:64] FLAG: --cluster-dns="[]"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958320    2563 flags.go:64] FLAG: --cluster-domain=""
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958323    2563 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958326    2563 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958329    2563 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 09 06:51:07.962629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958332    2563 flags.go:64] FLAG: --container-log-max-files="5"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958336    2563 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958339    2563 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958342    2563 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958346    2563 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958349    2563 flags.go:64] FLAG: --contention-profiling="false"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958352    2563 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958355    2563 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958358    2563 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958361    2563 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958365    2563 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958368    2563 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958371    2563 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958374    2563 flags.go:64] FLAG: --enable-load-reader="false"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958377    2563 flags.go:64] FLAG: --enable-server="true"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958380    2563 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958384    2563 flags.go:64] FLAG: --event-burst="100"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958388    2563 flags.go:64] FLAG: --event-qps="50"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958390    2563 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958393    2563 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958396    2563 flags.go:64] FLAG: --eviction-hard=""
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958400    2563 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958403    2563 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958405    2563 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958408    2563 flags.go:64] FLAG: --eviction-soft=""
Sep 09 06:51:07.963212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958411    2563 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958414    2563 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958417    2563 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958422    2563 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958424    2563 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958427    2563 flags.go:64] FLAG: --fail-swap-on="true"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958430    2563 flags.go:64] FLAG: --feature-gates=""
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958434    2563 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958437    2563 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958440    2563 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958444    2563 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958447    2563 flags.go:64] FLAG: --healthz-port="10248"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958450    2563 flags.go:64] FLAG: --help="false"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958453    2563 flags.go:64] FLAG: --hostname-override="ip-10-0-128-206.ec2.internal"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958456    2563 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958459    2563 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958462    2563 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958465    2563 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958469    2563 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958472    2563 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958475    2563 flags.go:64] FLAG: --image-service-endpoint=""
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958477    2563 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958480    2563 flags.go:64] FLAG: --kube-api-burst="100"
Sep 09 06:51:07.963817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958483    2563 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958486    2563 flags.go:64] FLAG: --kube-api-qps="50"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958489    2563 flags.go:64] FLAG: --kube-reserved=""
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958492    2563 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958495    2563 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958498    2563 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958501    2563 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958503    2563 flags.go:64] FLAG: --lock-file=""
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958507    2563 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958509    2563 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958512    2563 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958518    2563 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958523    2563 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958526    2563 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958529    2563 flags.go:64] FLAG: --logging-format="text"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958532    2563 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958535    2563 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958538    2563 flags.go:64] FLAG: --manifest-url=""
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958541    2563 flags.go:64] FLAG: --manifest-url-header=""
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958545    2563 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958548    2563 flags.go:64] FLAG: --max-open-files="1000000"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958552    2563 flags.go:64] FLAG: --max-pods="110"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958555    2563 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958558    2563 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958561    2563 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 09 06:51:07.964413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958564    2563 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958567    2563 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958570    2563 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958573    2563 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958581    2563 flags.go:64] FLAG: --node-status-max-images="50"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958597    2563 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958600    2563 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958604    2563 flags.go:64] FLAG: --pod-cidr=""
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958607    2563 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958612    2563 flags.go:64] FLAG: --pod-manifest-path=""
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958615    2563 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958618    2563 flags.go:64] FLAG: --pods-per-core="0"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958621    2563 flags.go:64] FLAG: --port="10250"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958624    2563 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958627    2563 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0b07399bccee9de35"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958630    2563 flags.go:64] FLAG: --qos-reserved=""
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958633    2563 flags.go:64] FLAG: --read-only-port="10255"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958636    2563 flags.go:64] FLAG: --register-node="true"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958639    2563 flags.go:64] FLAG: --register-with-taints=""
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958643    2563 flags.go:64] FLAG: --registry-burst="10"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958647    2563 flags.go:64] FLAG: --registry-qps="5"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958650    2563 flags.go:64] FLAG: --reserved-cpus=""
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958653    2563 flags.go:64] FLAG: --reserved-memory=""
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958657    2563 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958660    2563 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 09 06:51:07.965037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958663    2563 flags.go:64] FLAG: --rotate-certificates="false"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958666    2563 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958669    2563 flags.go:64] FLAG: --runonce="false"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958672    2563 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958674    2563 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958677    2563 flags.go:64] FLAG: --seccomp-default="false"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958680    2563 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958683    2563 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958686    2563 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958689    2563 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958692    2563 flags.go:64] FLAG: --storage-driver-password="root"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958696    2563 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958699    2563 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958702    2563 flags.go:64] FLAG: --storage-driver-user="root"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958704    2563 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958707    2563 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958710    2563 flags.go:64] FLAG: --system-cgroups=""
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958713    2563 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958718    2563 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958721    2563 flags.go:64] FLAG: --tls-cert-file=""
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958724    2563 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958728    2563 flags.go:64] FLAG: --tls-min-version=""
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958731    2563 flags.go:64] FLAG: --tls-private-key-file=""
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958734    2563 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958737    2563 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 09 06:51:07.965788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958740    2563 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958742    2563 flags.go:64] FLAG: --v="2"
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958747    2563 flags.go:64] FLAG: --version="false"
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958752    2563 flags.go:64] FLAG: --vmodule=""
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958755    2563 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.958758    2563 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958875    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958878    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958882    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958885    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958887    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958890    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958893    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958895    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958898    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958902    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958905    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958907    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958910    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958912    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958915    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 06:51:07.966393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958918    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958920    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958923    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958927    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958930    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958933    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958936    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958939    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958942    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958944    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958947    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958949    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958952    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958956    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958958    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958961    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958964    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958966    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958969    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 06:51:07.966988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958972    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958974    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958977    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958980    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958982    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958985    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958988    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958992    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958994    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958997    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.958999    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959002    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959004    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959007    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959009    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959012    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959014    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959017    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959019    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959022    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 06:51:07.967460 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959024    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959026    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959029    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959031    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959034    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959036    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959041    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959043    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959046    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959049    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959052    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959054    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959058    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959061    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959064    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959066    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959069    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959071    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959074    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 06:51:07.967968 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959079    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959081    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959084    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959086    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959090    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959092    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959095    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959097    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959100    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959102    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959106    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959110    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959113    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959116    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959118    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959121    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959124    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959126    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959129    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 06:51:07.968441 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959134    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959136    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959139    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959142    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959145    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959148    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.959150    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.959892    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.967836    2563 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.967854    2563 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.967874    2563 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.967880    2563 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967937    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967943    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967947    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 06:51:07.968928 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967950    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967954    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967957    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967960    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967962    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967966    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967969    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967971    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967975    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967977    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967980    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967983    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967986    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967989    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967991    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967994    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.967997    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968000    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968002    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968005    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 06:51:07.969298 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968007    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968010    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968013    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968016    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968018    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968021    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968025    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968028    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968030    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968033    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968036    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968038    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968041    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968044    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968046    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968049    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968051    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968054    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968057    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968059    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 06:51:07.969797 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968062    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968064    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968067    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968069    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968073    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968077    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968080    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968082    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968085    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968088    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968091    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968094    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968096    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968099    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968102    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968104    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968107    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968109    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968112    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 06:51:07.970284 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968115    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968117    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968120    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968122    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968125    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968128    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968130    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968133    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968135    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968138    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968141    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968143    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968146    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968148    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968151    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968153    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968157    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968160    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968163    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968166    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 06:51:07.970823 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968169    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968172    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968175    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968177    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968180    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968183    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968185    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968188    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968191    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968193    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968196    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968199    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968201    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968205    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968207    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968210    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 06:51:07.971335 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968212    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.968217    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968329    2563 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968333    2563 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968336    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968339    2563 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968342    2563 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968345    2563 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968348    2563 feature_gate.go:328] unrecognized feature gate: Example2
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968350    2563 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968353    2563 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968355    2563 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968358    2563 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968360    2563 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 09 06:51:07.971783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968363    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968366    2563 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968369    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968371    2563 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968374    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968376    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968379    2563 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968381    2563 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968384    2563 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968387    2563 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968389    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968392    2563 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968395    2563 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968397    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968400    2563 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968402    2563 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968405    2563 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968408    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968410    2563 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968413    2563 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 09 06:51:07.972124 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968415    2563 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968417    2563 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968420    2563 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968422    2563 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968425    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968427    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968430    2563 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968432    2563 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968435    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968437    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968440    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968442    2563 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968445    2563 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968448    2563 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968451    2563 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968453    2563 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968456    2563 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968459    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968461    2563 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968464    2563 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 09 06:51:07.972624 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968466    2563 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968469    2563 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968472    2563 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968474    2563 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968477    2563 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968479    2563 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968482    2563 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968484    2563 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968487    2563 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968490    2563 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968495    2563 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968497    2563 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968500    2563 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968503    2563 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968505    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968507    2563 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968510    2563 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968513    2563 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968516    2563 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 09 06:51:07.973110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968520    2563 feature_gate.go:328] unrecognized feature gate: Example
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968523    2563 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968526    2563 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968529    2563 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968531    2563 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968533    2563 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968536    2563 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968539    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968542    2563 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968544    2563 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968546    2563 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968549    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968551    2563 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968554    2563 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968557    2563 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968559    2563 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968562    2563 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968565    2563 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968567    2563 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968570    2563 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 09 06:51:07.973573 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968572    2563 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968576    2563 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968579    2563 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968598    2563 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968603    2563 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968607    2563 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968610    2563 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:07.968612    2563 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.968618    2563 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.968745    2563 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.972917    2563 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.973894    2563 server.go:1019] "Starting client certificate rotation"
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.973987    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 06:51:07.974068 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.974018    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 09 06:51:07.976734 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.976721    2563 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 06:51:07.981628 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.981606    2563 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 09 06:51:07.995000 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.994977    2563 log.go:25] "Validated CRI v1 runtime API"
Sep 09 06:51:07.998300 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.998282    2563 log.go:25] "Validated CRI v1 image API"
Sep 09 06:51:07.999186 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:07.999166    2563 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 09 06:51:08.001063 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.001045    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 06:51:08.003862 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.003845    2563 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 a5165f80-acde-4e72-a05b-9b1bc2a4e871:/dev/nvme0n1p4 da9ac961-0a32-4a0a-afc3-c4f6f332bdc4:/dev/nvme0n1p3]
Sep 09 06:51:08.003903 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.003862    2563 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 09 06:51:08.010566 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.010458    2563 manager.go:216] Machine: {Timestamp:2026-09-09 06:51:08.007430483 +0000 UTC m=+0.377689562 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3095936 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2ed4ccb2c3443364fd2a925839e41f SystemUUID:ec2ed4cc-b2c3-4433-64fd-2a925839e41f BootID:6da8ae3e-b391-4bc2-aeae-5dbe4c727c83 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:44:b0:df:05:e9 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:44:b0:df:05:e9 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:4d:53:16:e6:b0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 09 06:51:08.010566 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.010562    2563 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 09 06:51:08.010664 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.010655    2563 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 09 06:51:08.014087 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.014065    2563 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 09 06:51:08.014235 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.014091    2563 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-206.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 09 06:51:08.014846 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.014835    2563 topology_manager.go:138] "Creating topology manager with none policy"
Sep 09 06:51:08.014876 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.014849    2563 container_manager_linux.go:309] "Creating device plugin manager"
Sep 09 06:51:08.014876 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.014860    2563 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 06:51:08.015647 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.015637    2563 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 09 06:51:08.017354 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.017345    2563 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 09 06:51:08.021695 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.021684    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 06:51:08.021807 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.021798    2563 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 09 06:51:08.021838 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.021826    2563 kubelet.go:486] "Attempting to sync node with API server"
Sep 09 06:51:08.021838 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.021836    2563 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 09 06:51:08.021890 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.021849    2563 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 09 06:51:08.021890 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.021859    2563 kubelet.go:398] "Adding apiserver pod source"
Sep 09 06:51:08.021890 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.021868    2563 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 09 06:51:08.022263 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.022251    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 06:51:08.022319 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.022268    2563 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 09 06:51:08.023021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.023003    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2b7z4"
Sep 09 06:51:08.024743 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.024725    2563 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 09 06:51:08.027676 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.027660    2563 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 09 06:51:08.027778 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.027697    2563 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 09 06:51:08.027940 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.027926    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 09 06:51:08.028001 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.027946    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 09 06:51:08.028001 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.027954    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 09 06:51:08.028001 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.027963    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 09 06:51:08.028001 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.027974    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 09 06:51:08.028001 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.027983    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 09 06:51:08.028625 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.028615    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 09 06:51:08.028675 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.028627    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 09 06:51:08.028675 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.028637    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 09 06:51:08.028675 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.028645    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 09 06:51:08.028675 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.028664    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 09 06:51:08.028812 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.028686    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 09 06:51:08.029599 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.029576    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 09 06:51:08.029653 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.029606    2563 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 09 06:51:08.030923 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.030902    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2b7z4"
Sep 09 06:51:08.032641 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.032623    2563 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-206.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 09 06:51:08.033227 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.033208    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-206.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 09 06:51:08.033318 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.033210    2563 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 09 06:51:08.033635 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.033621    2563 server.go:1268] "Started kubelet"
Sep 09 06:51:08.033723 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.033691    2563 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 09 06:51:08.037138 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.036949    2563 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 09 06:51:08.037138 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.037035    2563 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 09 06:51:08.037363 ip-10-0-128-206 systemd[1]: Started Kubernetes Kubelet.
Sep 09 06:51:08.038696 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.038670    2563 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 09 06:51:08.039527 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.039437    2563 server.go:310] "Adding debug handlers to kubelet server"
Sep 09 06:51:08.039896 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.039877    2563 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 09 06:51:08.044521 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.044499    2563 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 09 06:51:08.044614 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.043507    2563 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-206.ec2.internal.18d3946bdfcfef0e  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-206.ec2.internal,UID:ip-10-0-128-206.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-206.ec2.internal,},FirstTimestamp:2026-09-09 06:51:08.033576718 +0000 UTC m=+0.403835796,LastTimestamp:2026-09-09 06:51:08.033576718 +0000 UTC m=+0.403835796,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-206.ec2.internal,}"
Sep 09 06:51:08.045214 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.045199    2563 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 09 06:51:08.045463 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.045288    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.046007 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.045987    2563 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 09 06:51:08.046007 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.045989    2563 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 09 06:51:08.046137 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046019    2563 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 09 06:51:08.046137 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046089    2563 reconstruct.go:97] "Volume reconstruction finished"
Sep 09 06:51:08.046137 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046099    2563 reconciler.go:29] "Reconciler: start to sync state"
Sep 09 06:51:08.046137 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046090    2563 factory.go:55] Registering systemd factory
Sep 09 06:51:08.046137 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046117    2563 factory.go:223] Registration of the systemd container factory successfully
Sep 09 06:51:08.046372 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046296    2563 factory.go:153] Registering CRI-O factory
Sep 09 06:51:08.046372 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046309    2563 factory.go:223] Registration of the crio container factory successfully
Sep 09 06:51:08.046372 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046350    2563 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 09 06:51:08.046372 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046363    2563 factory.go:103] Registering Raw factory
Sep 09 06:51:08.046544 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046382    2563 manager.go:1196] Started watching for new ooms in manager
Sep 09 06:51:08.046843 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.046828    2563 manager.go:349] Starting recovery of all containers
Sep 09 06:51:08.056455 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.056434    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 06:51:08.057362 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.057346    2563 manager.go:354] Recovery completed
Sep 09 06:51:08.059786 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.059766    2563 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-206.ec2.internal\" not found" node="ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.062596 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.062572    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 06:51:08.064879 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.064861    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 06:51:08.064961 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.064896    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 06:51:08.064961 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.064911    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientPID"
Sep 09 06:51:08.065571 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.065558    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 06:51:08.065665 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.065558    2563 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 09 06:51:08.065665 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.065612    2563 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 09 06:51:08.065665 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.065632    2563 state_mem.go:36] "Initialized new in-memory state store"
Sep 09 06:51:08.066272 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.066256    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 06:51:08.066338 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.066287    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 06:51:08.066338 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.066304    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientPID"
Sep 09 06:51:08.067846 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.067832    2563 policy_none.go:49] "None policy: Start"
Sep 09 06:51:08.067885 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.067851    2563 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 09 06:51:08.067885 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.067864    2563 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 09 06:51:08.069847 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.069836    2563 policy_none.go:47] "Start"
Sep 09 06:51:08.100301 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.100274    2563 manager.go:337] "Starting Device Plugin manager"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.100318    2563 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.100331    2563 server.go:87] "Starting device plugin registration server"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.100609    2563 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.100622    2563 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.100702    2563 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.100794    2563 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.100803    2563 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.101438    2563 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.101492    2563 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.108515    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.110103    2563 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.110143    2563 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.110168    2563 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.110205    2563 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 09 06:51:08.121180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.112524    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 06:51:08.201143 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.201039    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 06:51:08.202072 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.202048    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 06:51:08.202177 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.202087    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 06:51:08.202177 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.202100    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientPID"
Sep 09 06:51:08.202177 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.202131    2563 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.210943 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.210922    2563 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal"]
Sep 09 06:51:08.211029 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.210987    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 06:51:08.211552 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.211537    2563 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.211625 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.211562    2563 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-206.ec2.internal\": node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.211816 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.211802    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 06:51:08.211880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.211831    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 06:51:08.211880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.211845    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientPID"
Sep 09 06:51:08.214164 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214149    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 06:51:08.214298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214283    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.214352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214315    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 06:51:08.214872 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214854    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 06:51:08.214968 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214886    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 06:51:08.214968 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214899    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientPID"
Sep 09 06:51:08.214968 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214854    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 06:51:08.214968 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214952    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 06:51:08.214968 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.214964    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientPID"
Sep 09 06:51:08.217157 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.217142    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.217220 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.217168    2563 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 09 06:51:08.217811 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.217798    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientMemory"
Sep 09 06:51:08.217867 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.217819    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasNoDiskPressure"
Sep 09 06:51:08.217867 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.217828    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeHasSufficientPID"
Sep 09 06:51:08.232437 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.232422    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.246954 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.246937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29fe8aee47d6454f6b21baaf9f280874-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal\" (UID: \"29fe8aee47d6454f6b21baaf9f280874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.247039 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.246963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29fe8aee47d6454f6b21baaf9f280874-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal\" (UID: \"29fe8aee47d6454f6b21baaf9f280874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.247039 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.246978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05d7194320428eb238b4058701b24ef4-config\") pod \"kube-apiserver-proxy-ip-10-0-128-206.ec2.internal\" (UID: \"05d7194320428eb238b4058701b24ef4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.248330 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.248316    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-206.ec2.internal\" not found" node="ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.252791 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.252775    2563 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-206.ec2.internal\" not found" node="ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.332507 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.332476    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.347105 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.347081    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29fe8aee47d6454f6b21baaf9f280874-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal\" (UID: \"29fe8aee47d6454f6b21baaf9f280874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.347161 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.347109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29fe8aee47d6454f6b21baaf9f280874-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal\" (UID: \"29fe8aee47d6454f6b21baaf9f280874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.347161 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.347127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05d7194320428eb238b4058701b24ef4-config\") pod \"kube-apiserver-proxy-ip-10-0-128-206.ec2.internal\" (UID: \"05d7194320428eb238b4058701b24ef4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.347226 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.347165    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/05d7194320428eb238b4058701b24ef4-config\") pod \"kube-apiserver-proxy-ip-10-0-128-206.ec2.internal\" (UID: \"05d7194320428eb238b4058701b24ef4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.347226 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.347195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/29fe8aee47d6454f6b21baaf9f280874-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal\" (UID: \"29fe8aee47d6454f6b21baaf9f280874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.347226 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.347195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/29fe8aee47d6454f6b21baaf9f280874-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal\" (UID: \"29fe8aee47d6454f6b21baaf9f280874\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.432794 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.432760    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.533611 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.533529    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.552074 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.552044    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.557339 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.557315    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:08.634173 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.633991    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.734519 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.734492    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.835523 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.835494    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.939103 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:08.939072    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:08.973269 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.973225    2563 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 09 06:51:08.973825 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.973377    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 06:51:08.973825 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:08.973422    2563 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 09 06:51:09.038495 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.038459    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-08 06:46:09 +0000 UTC" deadline="2028-02-17 16:15:10.569693471 +0000 UTC"
Sep 09 06:51:09.038495 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.038487    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12633h24m1.53120951s"
Sep 09 06:51:09.039297 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:09.039278    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:09.140227 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:09.140126    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:09.168421 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:09.168388    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05d7194320428eb238b4058701b24ef4.slice/crio-4f040a9da0a6861223fdf574f3379a0a99914be06c8305eb851c16adb8218dbc WatchSource:0}: Error finding container 4f040a9da0a6861223fdf574f3379a0a99914be06c8305eb851c16adb8218dbc: Status 404 returned error can't find the container with id 4f040a9da0a6861223fdf574f3379a0a99914be06c8305eb851c16adb8218dbc
Sep 09 06:51:09.169069 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:09.169052    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29fe8aee47d6454f6b21baaf9f280874.slice/crio-0b9bf69b0d7a6ce9f967a21998fe3563904f5579465b9d00dc9428cb8f2b9f43 WatchSource:0}: Error finding container 0b9bf69b0d7a6ce9f967a21998fe3563904f5579465b9d00dc9428cb8f2b9f43: Status 404 returned error can't find the container with id 0b9bf69b0d7a6ce9f967a21998fe3563904f5579465b9d00dc9428cb8f2b9f43
Sep 09 06:51:09.177273 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.177252    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 06:51:09.240230 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:09.240194    2563 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-206.ec2.internal\" not found"
Sep 09 06:51:09.244904 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.244873    2563 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 06:51:09.246597 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.246583    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:09.254698 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.254674    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 06:51:09.257705 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.257689    2563 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal"
Sep 09 06:51:09.267730 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.267704    2563 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 09 06:51:09.374149 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:09.374111    2563 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 06:51:10.022995 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.022962    2563 apiserver.go:52] "Watching apiserver"
Sep 09 06:51:10.032518 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.032477    2563 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 09 06:51:10.032978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.032941    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-97ml8","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd","openshift-dns/node-resolver-rkmgz","openshift-multus/multus-7dhhp","openshift-network-operator/iptables-alerter-62jhc","openshift-ovn-kubernetes/ovnkube-node-6xnwt","kube-system/global-pull-secret-syncer-7px44","kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal","openshift-cluster-node-tuning-operator/tuned-rps4n","openshift-image-registry/node-ca-6r2sr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal","openshift-multus/multus-additional-cni-plugins-kzxzm","openshift-multus/network-metrics-daemon-dkrzk","openshift-network-diagnostics/network-check-target-984vs"]
Sep 09 06:51:10.035641 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.035608    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.035747 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.035689    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:10.037935 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.037915    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:10.038024 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.037992    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:10.040385 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.040358    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.042815 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.042795    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 09 06:51:10.042920 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.042851    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.043076 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.043054    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dwxjt\""
Sep 09 06:51:10.043076 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.043069    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 09 06:51:10.044773 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.044743    2563 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 09 06:51:10.045297 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.045271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.045435 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.045414    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 09 06:51:10.045680 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.045642    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 09 06:51:10.045788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.045727    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 09 06:51:10.045880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.045861    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kn26q\""
Sep 09 06:51:10.045957 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.045861    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 09 06:51:10.048374 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.048016    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 09 06:51:10.048374 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.048111    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 09 06:51:10.048374 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.048272    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-t7hvw\""
Sep 09 06:51:10.048561 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.048397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 09 06:51:10.048561 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.048461    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.050658 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.050610    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.050986 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.050895    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-2hhfc\""
Sep 09 06:51:10.051087 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.051059    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 09 06:51:10.051413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.051387    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 09 06:51:10.051413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.051405    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 09 06:51:10.051558 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.051484    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 09 06:51:10.051637 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.051595    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 09 06:51:10.051738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.051721    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 09 06:51:10.053075 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.053056    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:10.053462 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.053429    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-pfhz4\""
Sep 09 06:51:10.053712 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.053690    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 09 06:51:10.053943 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.053927    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 09 06:51:10.055593 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.055575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 09 06:51:10.055710 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.055612    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.055784 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.055739    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-lxk2j\""
Sep 09 06:51:10.056088 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.056070    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 09 06:51:10.057651 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.057629    2563 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 09 06:51:10.057833 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.057811    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-k8s-cni-cncf-io\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.057912 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.057848    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-conf-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.057912 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.057875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-multus-certs\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.058010 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.057931    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.058010 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.057963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.058010 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.057984    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jvhd\" (UniqueName: \"kubernetes.io/projected/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-kube-api-access-8jvhd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.058164 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058010    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/857657b9-98d6-4f08-84cc-c832fa9d79ab-kubelet-config\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.058164 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058044    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd5w4\" (UniqueName: \"kubernetes.io/projected/c5585130-4596-4fb1-8751-0dd1a062dc2e-kube-api-access-kd5w4\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:10.058164 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058113    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-cni-bin\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.058164 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058141    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-hostroot\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.058367 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058168    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 09 06:51:10.058367 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq74h\" (UniqueName: \"kubernetes.io/projected/7d928919-0da3-4cd5-88f6-d4227323bc6a-kube-api-access-wq74h\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.058367 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b82c5253-36f3-4e09-8467-3085d29c0c65-iptables-alerter-script\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.058367 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058307    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-run-netns\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.058810 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058784    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.058905 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d928919-0da3-4cd5-88f6-d4227323bc6a-cni-binary-copy\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.058963 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058907    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-etc-kubernetes\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.059012 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058969    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-node-log\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.059012 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.058995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-log-socket\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.059200 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059183    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovnkube-config\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.059264 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059223    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-cni-multus\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.059317 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059270    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 09 06:51:10.059505 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059485    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b82c5253-36f3-4e09-8467-3085d29c0c65-host-slash\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.059579 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059527    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-kubelet\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.059579 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059555    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-slash\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.059695 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059597    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-var-lib-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.059830 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059650    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-cni-bin\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.059913 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059825    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-qqkbb\""
Sep 09 06:51:10.059913 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-cni-netd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.059913 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059893    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/857657b9-98d6-4f08-84cc-c832fa9d79ab-dbus\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.060064 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.059937    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-hosts-file\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.060621 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.060584    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-system-cni-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.060747 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.060729    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.061625 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.061599    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 09 06:51:10.061832 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.061812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-cni-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.061906 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.061856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-kubelet\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.061906 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.061901    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-systemd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.062012 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.061933    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.062012 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.061974    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-tmp-dir\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.062113 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062036    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-cnibin\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.062113 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-socket-dir-parent\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.062249 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062194    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-etc-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.062353 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062309    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-systemd-units\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.062353 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062345    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-ovn\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.062525 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062377    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovn-node-metrics-cert\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.062525 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062419    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovnkube-script-lib\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.062525 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:10.062525 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-65zgb\" (UniqueName: \"kubernetes.io/projected/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-kube-api-access-65zgb\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.062525 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062505    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-netns\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.062785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-daemon-config\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.062785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062565    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rmwq\" (UniqueName: \"kubernetes.io/projected/b82c5253-36f3-4e09-8467-3085d29c0c65-kube-api-access-5rmwq\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.062785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.062594    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-env-overrides\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.063264 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.063239    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 09 06:51:10.063264 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.063257    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:10.063417 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.063346    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:10.063709 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.063663    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-psk5w\""
Sep 09 06:51:10.063709 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.063700    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 09 06:51:10.064106 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.064087    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 09 06:51:10.066640 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.066621    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.069088 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.069066    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 09 06:51:10.069191 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.069128    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-lh8xw\""
Sep 09 06:51:10.069191 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.069184    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 09 06:51:10.080838 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.080820    2563 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fprct"
Sep 09 06:51:10.088595 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.088574    2563 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 06:51:10.089996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.089972    2563 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fprct"
Sep 09 06:51:10.115541 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.115484    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal" event={"ID":"29fe8aee47d6454f6b21baaf9f280874","Type":"ContainerStarted","Data":"0b9bf69b0d7a6ce9f967a21998fe3563904f5579465b9d00dc9428cb8f2b9f43"}
Sep 09 06:51:10.116798 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.116767    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal" event={"ID":"05d7194320428eb238b4058701b24ef4","Type":"ContainerStarted","Data":"4f040a9da0a6861223fdf574f3379a0a99914be06c8305eb851c16adb8218dbc"}
Sep 09 06:51:10.147868 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.147833    2563 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 09 06:51:10.163108 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163068    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/857657b9-98d6-4f08-84cc-c832fa9d79ab-dbus\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.163250 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-system-cni-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163250 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-cni-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163250 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163193    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-kubelet\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163250 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163212    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-tuned\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.163250 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdcrd\" (UniqueName: \"kubernetes.io/projected/635184e6-443f-4df2-99e0-496ddc9fbe02-kube-api-access-tdcrd\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.163250 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163237    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-system-cni-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163268    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-system-cni-dir\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163294    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-kubelet\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163319    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-tmp-dir\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163339    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-cnibin\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/857657b9-98d6-4f08-84cc-c832fa9d79ab-dbus\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163363    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-cni-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163388    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-cnibin\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-socket-dir-parent\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/41f1fa06-a5b4-410b-8000-7b5ebbbf3a94-konnectivity-ca\") pod \"konnectivity-agent-97ml8\" (UID: \"41f1fa06-a5b4-410b-8000-7b5ebbbf3a94\") " pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163476    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163514    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-socket-dir-parent\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-systemd-units\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.163556 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163561    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-systemd-units\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovn-node-metrics-cert\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163651    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovnkube-script-lib\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163680    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-systemd\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163689    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-tmp-dir\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163800    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-lib-modules\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163897    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/41f1fa06-a5b4-410b-8000-7b5ebbbf3a94-agent-certs\") pod \"konnectivity-agent-97ml8\" (UID: \"41f1fa06-a5b4-410b-8000-7b5ebbbf3a94\") " pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163932    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-netns\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163963    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rmwq\" (UniqueName: \"kubernetes.io/projected/b82c5253-36f3-4e09-8467-3085d29c0c65-kube-api-access-5rmwq\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.163991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-sys\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v8n87\" (UniqueName: \"kubernetes.io/projected/7c31d1b6-7168-45b0-b317-e4bc606fedcf-kube-api-access-v8n87\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164033    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-netns\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164045    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-device-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164072    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv2r9\" (UniqueName: \"kubernetes.io/projected/595a451a-b092-43db-91e4-ee3012869077-kube-api-access-mv2r9\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-k8s-cni-cncf-io\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-multus-certs\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.164298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164181    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164213    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164245    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysctl-d\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164275    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/857657b9-98d6-4f08-84cc-c832fa9d79ab-kubelet-config\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164302    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-cni-bin\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164329    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-hostroot\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164357    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b82c5253-36f3-4e09-8467-3085d29c0c65-iptables-alerter-script\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164388    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysconfig\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164451    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-registration-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164478    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-node-log\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164480    2563 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164508    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-log-socket\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164528    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-k8s-cni-cncf-io\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164539    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-kubernetes\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164552    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovnkube-script-lib\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164589    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-run-multus-certs\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164591    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-cnibin\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.165170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164658    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164681    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6rjr\" (UniqueName: \"kubernetes.io/projected/4b30aa0c-9473-4ad8-a676-e12441e128cf-kube-api-access-p6rjr\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164743    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164742    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-kubelet\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164780    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-kubelet\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-sys-fs\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164798    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-cni-bin\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164830    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-node-log\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164839    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-hosts-file\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-systemd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/857657b9-98d6-4f08-84cc-c832fa9d79ab-kubelet-config\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164876    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-log-socket\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164920    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-hostroot\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164947    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164955    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-systemd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-hosts-file\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.164982    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165029    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-etc-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.165096    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165136    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-etc-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165098    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-modprobe-d\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.165180    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret podName:857657b9-98d6-4f08-84cc-c832fa9d79ab nodeName:}" failed. No retries permitted until 2026-09-09 06:51:10.665143163 +0000 UTC m=+3.035402243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret") pod "global-pull-secret-syncer-7px44" (UID: "857657b9-98d6-4f08-84cc-c832fa9d79ab") : object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165226    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/635184e6-443f-4df2-99e0-496ddc9fbe02-serviceca\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-ovn\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165285    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-ovn\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-host\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165343    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c31d1b6-7168-45b0-b317-e4bc606fedcf-tmp\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-65zgb\" (UniqueName: \"kubernetes.io/projected/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-kube-api-access-65zgb\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165389    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b82c5253-36f3-4e09-8467-3085d29c0c65-iptables-alerter-script\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165420    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-daemon-config\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165445    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-env-overrides\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.166924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165471    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-socket-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-conf-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165511    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jvhd\" (UniqueName: \"kubernetes.io/projected/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-kube-api-access-8jvhd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165534    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/635184e6-443f-4df2-99e0-496ddc9fbe02-host\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.165543    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd5w4\" (UniqueName: \"kubernetes.io/projected/c5585130-4596-4fb1-8751-0dd1a062dc2e-kube-api-access-kd5w4\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165573    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq74h\" (UniqueName: \"kubernetes.io/projected/7d928919-0da3-4cd5-88f6-d4227323bc6a-kube-api-access-wq74h\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.165590    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs podName:c5585130-4596-4fb1-8751-0dd1a062dc2e nodeName:}" failed. No retries permitted until 2026-09-09 06:51:10.665575856 +0000 UTC m=+3.035834929 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs") pod "network-metrics-daemon-dkrzk" (UID: "c5585130-4596-4fb1-8751-0dd1a062dc2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165631    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-run-netns\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165677    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovnkube-config\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165699    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysctl-conf\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165720    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d928919-0da3-4cd5-88f6-d4227323bc6a-cni-binary-copy\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165776    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-etc-kubernetes\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-etc-kubernetes\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165826    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-run-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165851    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.167642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165884    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-cni-multus\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165914    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b82c5253-36f3-4e09-8467-3085d29c0c65-host-slash\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-slash\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.165987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-var-lib-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166012    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-cni-bin\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166040    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-cni-netd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-run\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166111    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-var-lib-kubelet\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166167    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b82c5253-36f3-4e09-8467-3085d29c0c65-host-slash\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166206    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-daemon-config\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166208    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-slash\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166232    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-run-netns\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166243    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-var-lib-openvswitch\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166275    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-multus-conf-dir\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166292    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-cni-bin\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166312    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovnkube-config\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-host-cni-netd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166500    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/7d928919-0da3-4cd5-88f6-d4227323bc6a-host-var-lib-cni-multus\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.168389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-env-overrides\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.169272 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.166890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/7d928919-0da3-4cd5-88f6-d4227323bc6a-cni-binary-copy\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.169272 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.168703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-ovn-node-metrics-cert\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.189691 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.188882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rmwq\" (UniqueName: \"kubernetes.io/projected/b82c5253-36f3-4e09-8467-3085d29c0c65-kube-api-access-5rmwq\") pod \"iptables-alerter-62jhc\" (UID: \"b82c5253-36f3-4e09-8467-3085d29c0c65\") " pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.189691 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.189403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jvhd\" (UniqueName: \"kubernetes.io/projected/931199f9-eaba-4aec-9bf0-5ce95a9d37b0-kube-api-access-8jvhd\") pod \"ovnkube-node-6xnwt\" (UID: \"931199f9-eaba-4aec-9bf0-5ce95a9d37b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.189911 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.189795    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd5w4\" (UniqueName: \"kubernetes.io/projected/c5585130-4596-4fb1-8751-0dd1a062dc2e-kube-api-access-kd5w4\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:10.191572 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.191216    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-65zgb\" (UniqueName: \"kubernetes.io/projected/11d5f98d-2bb0-4703-9b3a-2d72d8690e35-kube-api-access-65zgb\") pod \"node-resolver-rkmgz\" (UID: \"11d5f98d-2bb0-4703-9b3a-2d72d8690e35\") " pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.197031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.197003    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq74h\" (UniqueName: \"kubernetes.io/projected/7d928919-0da3-4cd5-88f6-d4227323bc6a-kube-api-access-wq74h\") pod \"multus-7dhhp\" (UID: \"7d928919-0da3-4cd5-88f6-d4227323bc6a\") " pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.266475 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266437    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-modprobe-d\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.266688 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/635184e6-443f-4df2-99e0-496ddc9fbe02-serviceca\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.266688 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-host\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.266688 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266579    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-host\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.266688 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266584    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c31d1b6-7168-45b0-b317-e4bc606fedcf-tmp\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.266688 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-modprobe-d\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.266688 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-socket-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266708    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/635184e6-443f-4df2-99e0-496ddc9fbe02-host\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266738    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysctl-conf\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266767    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-socket-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/635184e6-443f-4df2-99e0-496ddc9fbe02-host\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266855    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-run\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266798    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-run\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-var-lib-kubelet\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266938    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-tuned\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266964    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdcrd\" (UniqueName: \"kubernetes.io/projected/635184e6-443f-4df2-99e0-496ddc9fbe02-kube-api-access-tdcrd\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266979    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/635184e6-443f-4df2-99e0-496ddc9fbe02-serviceca\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-system-cni-dir\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.266995    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysctl-conf\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267024    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-system-cni-dir\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267038    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-kubelet-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.267086 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267063    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-var-lib-kubelet\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267123    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/41f1fa06-a5b4-410b-8000-7b5ebbbf3a94-konnectivity-ca\") pod \"konnectivity-agent-97ml8\" (UID: \"41f1fa06-a5b4-410b-8000-7b5ebbbf3a94\") " pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267191    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-systemd\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267234    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-lib-modules\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267249    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/41f1fa06-a5b4-410b-8000-7b5ebbbf3a94-agent-certs\") pod \"konnectivity-agent-97ml8\" (UID: \"41f1fa06-a5b4-410b-8000-7b5ebbbf3a94\") " pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267266    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-sys\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-systemd\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267355    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-lib-modules\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267367    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v8n87\" (UniqueName: \"kubernetes.io/projected/7c31d1b6-7168-45b0-b317-e4bc606fedcf-kube-api-access-v8n87\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-etc-selinux\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267736    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-sys\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-device-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/41f1fa06-a5b4-410b-8000-7b5ebbbf3a94-konnectivity-ca\") pod \"konnectivity-agent-97ml8\" (UID: \"41f1fa06-a5b4-410b-8000-7b5ebbbf3a94\") " pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267902    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-device-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.267919 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267877    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv2r9\" (UniqueName: \"kubernetes.io/projected/595a451a-b092-43db-91e4-ee3012869077-kube-api-access-mv2r9\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267842    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.267971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268008    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysctl-d\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysconfig\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268065    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-registration-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268115    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-kubernetes\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268123    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysconfig\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-cnibin\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268170    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6rjr\" (UniqueName: \"kubernetes.io/projected/4b30aa0c-9473-4ad8-a676-e12441e128cf-kube-api-access-p6rjr\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268215    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-kubernetes\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-sys-fs\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268257    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268269    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-registration-dir\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268286    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:10.268817 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268170    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-sysctl-d\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.269609 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268378    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/595a451a-b092-43db-91e4-ee3012869077-sys-fs\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.269609 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268417    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4b30aa0c-9473-4ad8-a676-e12441e128cf-cnibin\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.269609 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.268945    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.269609 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.269073    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4b30aa0c-9473-4ad8-a676-e12441e128cf-cni-binary-copy\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.269609 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.269545    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7c31d1b6-7168-45b0-b317-e4bc606fedcf-tmp\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.269892 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.269674    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7c31d1b6-7168-45b0-b317-e4bc606fedcf-etc-tuned\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.270416 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.270391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/41f1fa06-a5b4-410b-8000-7b5ebbbf3a94-agent-certs\") pod \"konnectivity-agent-97ml8\" (UID: \"41f1fa06-a5b4-410b-8000-7b5ebbbf3a94\") " pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:10.275088 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.275027    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 06:51:10.275088 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.275058    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 06:51:10.275088 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.275077    2563 projected.go:196] Error preparing data for projected volume kube-api-access-l25wb for pod openshift-network-diagnostics/network-check-target-984vs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:10.275388 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.275156    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb podName:2c131071-1759-46a1-9f59-d55efbbab153 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:10.775136035 +0000 UTC m=+3.145395125 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l25wb" (UniqueName: "kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb") pod "network-check-target-984vs" (UID: "2c131071-1759-46a1-9f59-d55efbbab153") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:10.276213 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.276189    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdcrd\" (UniqueName: \"kubernetes.io/projected/635184e6-443f-4df2-99e0-496ddc9fbe02-kube-api-access-tdcrd\") pod \"node-ca-6r2sr\" (UID: \"635184e6-443f-4df2-99e0-496ddc9fbe02\") " pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.276859 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.276835    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6rjr\" (UniqueName: \"kubernetes.io/projected/4b30aa0c-9473-4ad8-a676-e12441e128cf-kube-api-access-p6rjr\") pod \"multus-additional-cni-plugins-kzxzm\" (UID: \"4b30aa0c-9473-4ad8-a676-e12441e128cf\") " pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.277352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.277328    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv2r9\" (UniqueName: \"kubernetes.io/projected/595a451a-b092-43db-91e4-ee3012869077-kube-api-access-mv2r9\") pod \"aws-ebs-csi-driver-node-wwhmd\" (UID: \"595a451a-b092-43db-91e4-ee3012869077\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.278948 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.278929    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v8n87\" (UniqueName: \"kubernetes.io/projected/7c31d1b6-7168-45b0-b317-e4bc606fedcf-kube-api-access-v8n87\") pod \"tuned-rps4n\" (UID: \"7c31d1b6-7168-45b0-b317-e4bc606fedcf\") " pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.352338 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.352295    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rkmgz"
Sep 09 06:51:10.361376 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.361346    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-7dhhp"
Sep 09 06:51:10.372738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.372711    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-62jhc"
Sep 09 06:51:10.376448 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.376428    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:10.384238 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.384214    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-rps4n"
Sep 09 06:51:10.391255 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.391236    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:10.398101 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.398083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd"
Sep 09 06:51:10.406829 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.406807    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-6r2sr"
Sep 09 06:51:10.412559 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.412534    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-kzxzm"
Sep 09 06:51:10.418849 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.418831    2563 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 09 06:51:10.507694 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.507655    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb82c5253_36f3_4e09_8467_3085d29c0c65.slice/crio-4b4c92e6958a23b83515488aa5fca165370ef5d17c2213b1616ebbb393de29f5 WatchSource:0}: Error finding container 4b4c92e6958a23b83515488aa5fca165370ef5d17c2213b1616ebbb393de29f5: Status 404 returned error can't find the container with id 4b4c92e6958a23b83515488aa5fca165370ef5d17c2213b1616ebbb393de29f5
Sep 09 06:51:10.508896 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.508847    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c31d1b6_7168_45b0_b317_e4bc606fedcf.slice/crio-d7fb9d6f88706b860d62c87673cb81011d88949e5e7bcaabf90d73c131276ec5 WatchSource:0}: Error finding container d7fb9d6f88706b860d62c87673cb81011d88949e5e7bcaabf90d73c131276ec5: Status 404 returned error can't find the container with id d7fb9d6f88706b860d62c87673cb81011d88949e5e7bcaabf90d73c131276ec5
Sep 09 06:51:10.509860 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.509825    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod931199f9_eaba_4aec_9bf0_5ce95a9d37b0.slice/crio-768b6aa017b28fd2a1de3fd0b78c1f86bd1e909f7e4ddd331a369602d164948a WatchSource:0}: Error finding container 768b6aa017b28fd2a1de3fd0b78c1f86bd1e909f7e4ddd331a369602d164948a: Status 404 returned error can't find the container with id 768b6aa017b28fd2a1de3fd0b78c1f86bd1e909f7e4ddd331a369602d164948a
Sep 09 06:51:10.511387 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.511277    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41f1fa06_a5b4_410b_8000_7b5ebbbf3a94.slice/crio-8fb05619bbc9fdf3534651c853935f14256067822c8160ee3879909e38e04010 WatchSource:0}: Error finding container 8fb05619bbc9fdf3534651c853935f14256067822c8160ee3879909e38e04010: Status 404 returned error can't find the container with id 8fb05619bbc9fdf3534651c853935f14256067822c8160ee3879909e38e04010
Sep 09 06:51:10.513988 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.513966    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod595a451a_b092_43db_91e4_ee3012869077.slice/crio-b2ef3a0c3e88e74d07477960bd8c2ee99de82c47b83ce8dd4a18a1756831628d WatchSource:0}: Error finding container b2ef3a0c3e88e74d07477960bd8c2ee99de82c47b83ce8dd4a18a1756831628d: Status 404 returned error can't find the container with id b2ef3a0c3e88e74d07477960bd8c2ee99de82c47b83ce8dd4a18a1756831628d
Sep 09 06:51:10.515110 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.515068    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d928919_0da3_4cd5_88f6_d4227323bc6a.slice/crio-51cb68d4653444b3f6735befc9b6acbccab5b27e58e79b64f7bb8fff567eac0a WatchSource:0}: Error finding container 51cb68d4653444b3f6735befc9b6acbccab5b27e58e79b64f7bb8fff567eac0a: Status 404 returned error can't find the container with id 51cb68d4653444b3f6735befc9b6acbccab5b27e58e79b64f7bb8fff567eac0a
Sep 09 06:51:10.515836 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.515804    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b30aa0c_9473_4ad8_a676_e12441e128cf.slice/crio-29ae19db6c75a0fdea34c8e1eebd7d83f5a440203d3554c0150a5c25fa767809 WatchSource:0}: Error finding container 29ae19db6c75a0fdea34c8e1eebd7d83f5a440203d3554c0150a5c25fa767809: Status 404 returned error can't find the container with id 29ae19db6c75a0fdea34c8e1eebd7d83f5a440203d3554c0150a5c25fa767809
Sep 09 06:51:10.538625 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.538571    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d5f98d_2bb0_4703_9b3a_2d72d8690e35.slice/crio-131e5cb5a40f4c0e0abce8f82e465d276e5fa7e2f7e2a795a45dbcda198da04d WatchSource:0}: Error finding container 131e5cb5a40f4c0e0abce8f82e465d276e5fa7e2f7e2a795a45dbcda198da04d: Status 404 returned error can't find the container with id 131e5cb5a40f4c0e0abce8f82e465d276e5fa7e2f7e2a795a45dbcda198da04d
Sep 09 06:51:10.539859 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:10.539714    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod635184e6_443f_4df2_99e0_496ddc9fbe02.slice/crio-06aa6bf48ac300079a127695971dcb4dbf388287b10717d8594b66276fa2c879 WatchSource:0}: Error finding container 06aa6bf48ac300079a127695971dcb4dbf388287b10717d8594b66276fa2c879: Status 404 returned error can't find the container with id 06aa6bf48ac300079a127695971dcb4dbf388287b10717d8594b66276fa2c879
Sep 09 06:51:10.673253 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.673221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:10.673436 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.673265    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:10.673436 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.673390    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:10.673565 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.673391    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:10.673565 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.673453    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs podName:c5585130-4596-4fb1-8751-0dd1a062dc2e nodeName:}" failed. No retries permitted until 2026-09-09 06:51:11.673433916 +0000 UTC m=+4.043692987 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs") pod "network-metrics-daemon-dkrzk" (UID: "c5585130-4596-4fb1-8751-0dd1a062dc2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:10.673565 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.673543    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret podName:857657b9-98d6-4f08-84cc-c832fa9d79ab nodeName:}" failed. No retries permitted until 2026-09-09 06:51:11.673524958 +0000 UTC m=+4.043784041 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret") pod "global-pull-secret-syncer-7px44" (UID: "857657b9-98d6-4f08-84cc-c832fa9d79ab") : object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:10.875151 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:10.875122    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:10.875340 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.875314    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 06:51:10.875340 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.875342    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 06:51:10.875543 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.875353    2563 projected.go:196] Error preparing data for projected volume kube-api-access-l25wb for pod openshift-network-diagnostics/network-check-target-984vs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:10.875543 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:10.875412    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb podName:2c131071-1759-46a1-9f59-d55efbbab153 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:11.875394499 +0000 UTC m=+4.245653572 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l25wb" (UniqueName: "kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb") pod "network-check-target-984vs" (UID: "2c131071-1759-46a1-9f59-d55efbbab153") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:11.093902 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.093769    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 06:46:10 +0000 UTC" deadline="2028-03-06 20:02:10.827873938 +0000 UTC"
Sep 09 06:51:11.093902 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.093848    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13069h10m59.734030258s"
Sep 09 06:51:11.121294 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.121162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkmgz" event={"ID":"11d5f98d-2bb0-4703-9b3a-2d72d8690e35","Type":"ContainerStarted","Data":"131e5cb5a40f4c0e0abce8f82e465d276e5fa7e2f7e2a795a45dbcda198da04d"}
Sep 09 06:51:11.123135 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.123043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6r2sr" event={"ID":"635184e6-443f-4df2-99e0-496ddc9fbe02","Type":"ContainerStarted","Data":"06aa6bf48ac300079a127695971dcb4dbf388287b10717d8594b66276fa2c879"}
Sep 09 06:51:11.125026 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.124955    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7dhhp" event={"ID":"7d928919-0da3-4cd5-88f6-d4227323bc6a","Type":"ContainerStarted","Data":"51cb68d4653444b3f6735befc9b6acbccab5b27e58e79b64f7bb8fff567eac0a"}
Sep 09 06:51:11.126629 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.126578    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd" event={"ID":"595a451a-b092-43db-91e4-ee3012869077","Type":"ContainerStarted","Data":"b2ef3a0c3e88e74d07477960bd8c2ee99de82c47b83ce8dd4a18a1756831628d"}
Sep 09 06:51:11.129079 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.129011    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-97ml8" event={"ID":"41f1fa06-a5b4-410b-8000-7b5ebbbf3a94","Type":"ContainerStarted","Data":"8fb05619bbc9fdf3534651c853935f14256067822c8160ee3879909e38e04010"}
Sep 09 06:51:11.130466 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.130387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-62jhc" event={"ID":"b82c5253-36f3-4e09-8467-3085d29c0c65","Type":"ContainerStarted","Data":"4b4c92e6958a23b83515488aa5fca165370ef5d17c2213b1616ebbb393de29f5"}
Sep 09 06:51:11.134079 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.134054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal" event={"ID":"29fe8aee47d6454f6b21baaf9f280874","Type":"ContainerStarted","Data":"dae76170a32d3d65e7e02f448108e474d143dede64054a2c9839653a640a8660"}
Sep 09 06:51:11.136904 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.136882    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" event={"ID":"4b30aa0c-9473-4ad8-a676-e12441e128cf","Type":"ContainerStarted","Data":"29ae19db6c75a0fdea34c8e1eebd7d83f5a440203d3554c0150a5c25fa767809"}
Sep 09 06:51:11.140117 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.140033    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"768b6aa017b28fd2a1de3fd0b78c1f86bd1e909f7e4ddd331a369602d164948a"}
Sep 09 06:51:11.142161 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.142099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rps4n" event={"ID":"7c31d1b6-7168-45b0-b317-e4bc606fedcf","Type":"ContainerStarted","Data":"d7fb9d6f88706b860d62c87673cb81011d88949e5e7bcaabf90d73c131276ec5"}
Sep 09 06:51:11.682939 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.682895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:11.683126 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.682960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:11.683126 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:11.683106    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:11.683249 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:11.683172    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs podName:c5585130-4596-4fb1-8751-0dd1a062dc2e nodeName:}" failed. No retries permitted until 2026-09-09 06:51:13.683152212 +0000 UTC m=+6.053411296 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs") pod "network-metrics-daemon-dkrzk" (UID: "c5585130-4596-4fb1-8751-0dd1a062dc2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:11.683606 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:11.683580    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:11.683747 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:11.683665    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret podName:857657b9-98d6-4f08-84cc-c832fa9d79ab nodeName:}" failed. No retries permitted until 2026-09-09 06:51:13.683648931 +0000 UTC m=+6.053908007 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret") pod "global-pull-secret-syncer-7px44" (UID: "857657b9-98d6-4f08-84cc-c832fa9d79ab") : object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:11.885128 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:11.885079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:11.885351 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:11.885309    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 06:51:11.885351 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:11.885330    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 06:51:11.885351 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:11.885344    2563 projected.go:196] Error preparing data for projected volume kube-api-access-l25wb for pod openshift-network-diagnostics/network-check-target-984vs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:11.885527 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:11.885408    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb podName:2c131071-1759-46a1-9f59-d55efbbab153 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:13.885388741 +0000 UTC m=+6.255647826 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-l25wb" (UniqueName: "kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb") pod "network-check-target-984vs" (UID: "2c131071-1759-46a1-9f59-d55efbbab153") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:12.094981 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:12.094833    2563 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-08 06:46:10 +0000 UTC" deadline="2028-02-25 16:12:57.954857415 +0000 UTC"
Sep 09 06:51:12.094981 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:12.094877    2563 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12825h21m45.859985789s"
Sep 09 06:51:12.111085 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:12.110511    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:12.111085 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:12.110653    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:12.111309 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:12.111092    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:12.111309 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:12.111183    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:12.111309 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:12.111255    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:12.111474 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:12.111343    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:13.154210 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:13.153931    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal" event={"ID":"05d7194320428eb238b4058701b24ef4","Type":"ContainerStarted","Data":"0ba3f0efcf4127695ed8946c39aec35bc209bb8ac0c2b01cba9a0690dfc0a422"}
Sep 09 06:51:13.160808 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:13.160701    2563 generic.go:356] "Generic (PLEG): container finished" podID="29fe8aee47d6454f6b21baaf9f280874" containerID="dae76170a32d3d65e7e02f448108e474d143dede64054a2c9839653a640a8660" exitCode=0
Sep 09 06:51:13.160808 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:13.160749    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal" event={"ID":"29fe8aee47d6454f6b21baaf9f280874","Type":"ContainerDied","Data":"dae76170a32d3d65e7e02f448108e474d143dede64054a2c9839653a640a8660"}
Sep 09 06:51:13.191313 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:13.190679    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-206.ec2.internal" podStartSLOduration=3.190636935 podStartE2EDuration="3.190636935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:13.172304007 +0000 UTC m=+5.542563100" watchObservedRunningTime="2026-09-09 06:51:13.190636935 +0000 UTC m=+5.560896030"
Sep 09 06:51:13.702831 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:13.702513    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:13.702831 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:13.702569    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:13.702831 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:13.702720    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:13.702831 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:13.702722    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:13.702831 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:13.702783    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs podName:c5585130-4596-4fb1-8751-0dd1a062dc2e nodeName:}" failed. No retries permitted until 2026-09-09 06:51:17.702762837 +0000 UTC m=+10.073021907 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs") pod "network-metrics-daemon-dkrzk" (UID: "c5585130-4596-4fb1-8751-0dd1a062dc2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:13.702831 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:13.702803    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret podName:857657b9-98d6-4f08-84cc-c832fa9d79ab nodeName:}" failed. No retries permitted until 2026-09-09 06:51:17.702793224 +0000 UTC m=+10.073052295 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret") pod "global-pull-secret-syncer-7px44" (UID: "857657b9-98d6-4f08-84cc-c832fa9d79ab") : object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:13.904675 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:13.904604    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:13.904856 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:13.904815    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 06:51:13.904856 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:13.904841    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 06:51:13.904856 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:13.904855    2563 projected.go:196] Error preparing data for projected volume kube-api-access-l25wb for pod openshift-network-diagnostics/network-check-target-984vs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:13.905043 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:13.904926    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb podName:2c131071-1759-46a1-9f59-d55efbbab153 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:17.904904754 +0000 UTC m=+10.275163838 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-l25wb" (UniqueName: "kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb") pod "network-check-target-984vs" (UID: "2c131071-1759-46a1-9f59-d55efbbab153") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:14.112778 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:14.110885    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:14.112778 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:14.111029    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:14.112778 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:14.111432    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:14.112778 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:14.111539    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:14.112778 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:14.111589    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:14.112778 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:14.111679    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:16.110600 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:16.110561    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:16.111116 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:16.110726    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:16.111116 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:16.110774    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:16.111116 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:16.110904    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:16.111116 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:16.110776    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:16.111116 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:16.111023    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:17.739847 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:17.739809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:17.740267 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:17.739865    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:17.740267 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:17.739995    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:17.740267 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:17.740065    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs podName:c5585130-4596-4fb1-8751-0dd1a062dc2e nodeName:}" failed. No retries permitted until 2026-09-09 06:51:25.740041782 +0000 UTC m=+18.110300855 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs") pod "network-metrics-daemon-dkrzk" (UID: "c5585130-4596-4fb1-8751-0dd1a062dc2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:17.740510 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:17.740474    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:17.740660 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:17.740543    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret podName:857657b9-98d6-4f08-84cc-c832fa9d79ab nodeName:}" failed. No retries permitted until 2026-09-09 06:51:25.740526837 +0000 UTC m=+18.110785920 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret") pod "global-pull-secret-syncer-7px44" (UID: "857657b9-98d6-4f08-84cc-c832fa9d79ab") : object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:17.941958 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:17.941911    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:17.942142 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:17.942122    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 06:51:17.942228 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:17.942147    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 06:51:17.942228 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:17.942161    2563 projected.go:196] Error preparing data for projected volume kube-api-access-l25wb for pod openshift-network-diagnostics/network-check-target-984vs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:17.942335 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:17.942229    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb podName:2c131071-1759-46a1-9f59-d55efbbab153 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:25.942209346 +0000 UTC m=+18.312468420 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-l25wb" (UniqueName: "kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb") pod "network-check-target-984vs" (UID: "2c131071-1759-46a1-9f59-d55efbbab153") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:18.111621 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:18.111525    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:18.111621 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:18.111618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:18.111880 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:18.111645    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:18.111880 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:18.111707    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:18.111880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:18.111752    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:18.111880 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:18.111829    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:20.114996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:20.114964    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:20.114996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:20.114995    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:20.115544 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:20.115078    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:20.115544 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:20.115109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:20.115544 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:20.115204    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:20.115544 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:20.115288    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:22.113961 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:22.113891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:22.114387 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:22.113892    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:22.114387 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:22.114011    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:22.114387 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:22.113893    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:22.114387 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:22.114086    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:22.114387 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:22.114180    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:24.110775 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:24.110706    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:24.111243 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:24.110706    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:24.111243 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:24.110838    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:24.111243 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:24.110833    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:24.111243 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:24.110925    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:24.111243 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:24.111000    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:25.804541 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:25.804500    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:25.804992 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:25.804609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:25.804992 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:25.804693    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:25.804992 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:25.804771    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs podName:c5585130-4596-4fb1-8751-0dd1a062dc2e nodeName:}" failed. No retries permitted until 2026-09-09 06:51:41.804752204 +0000 UTC m=+34.175011285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs") pod "network-metrics-daemon-dkrzk" (UID: "c5585130-4596-4fb1-8751-0dd1a062dc2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:25.804992 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:25.804698    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:25.804992 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:25.804846    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret podName:857657b9-98d6-4f08-84cc-c832fa9d79ab nodeName:}" failed. No retries permitted until 2026-09-09 06:51:41.804828585 +0000 UTC m=+34.175087653 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret") pod "global-pull-secret-syncer-7px44" (UID: "857657b9-98d6-4f08-84cc-c832fa9d79ab") : object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:26.006029 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:26.005986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:26.006193 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:26.006172    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 06:51:26.006247 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:26.006202    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 06:51:26.006247 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:26.006216    2563 projected.go:196] Error preparing data for projected volume kube-api-access-l25wb for pod openshift-network-diagnostics/network-check-target-984vs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:26.006352 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:26.006277    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb podName:2c131071-1759-46a1-9f59-d55efbbab153 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:42.006263378 +0000 UTC m=+34.376522448 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-l25wb" (UniqueName: "kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb") pod "network-check-target-984vs" (UID: "2c131071-1759-46a1-9f59-d55efbbab153") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:26.110840 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:26.110749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:26.110840 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:26.110760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:26.111044 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:26.110877    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:26.111044 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:26.110760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:26.111044 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:26.110937    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:26.111044 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:26.111012    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:28.112204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:28.112030    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:28.112204 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:28.112155    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:28.112617 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:28.112517    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:28.112671 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:28.112626    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:28.112671 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:28.112653    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:28.112843 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:28.112750    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:30.111116 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.110770    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:30.111964 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.110811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:30.111964 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:30.111169    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:30.111964 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.110845    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:30.111964 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:30.111214    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:30.111964 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:30.111289    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:30.210988 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.210951    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rkmgz" event={"ID":"11d5f98d-2bb0-4703-9b3a-2d72d8690e35","Type":"ContainerStarted","Data":"ddc519c1423a0b92fcc99c3509313c6150e0bfb1823f96e07ab286a463092357"}
Sep 09 06:51:30.212716 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.212689    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-6r2sr" event={"ID":"635184e6-443f-4df2-99e0-496ddc9fbe02","Type":"ContainerStarted","Data":"e7ee18ce155afb7e82867d9470f432d887173f321e6860593192d1b4be8d4f5c"}
Sep 09 06:51:30.214432 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.214403    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-7dhhp" event={"ID":"7d928919-0da3-4cd5-88f6-d4227323bc6a","Type":"ContainerStarted","Data":"24c1b4a13cd86071e64befe7c08d7019cbd72b3bb2fbe8b8eb6c889c0bedb506"}
Sep 09 06:51:30.216244 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.216223    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd" event={"ID":"595a451a-b092-43db-91e4-ee3012869077","Type":"ContainerStarted","Data":"47a17fca4b908c24d7974140aeb7aba4d9659c854d5823497464131930576c45"}
Sep 09 06:51:30.217793 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.217773    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-97ml8" event={"ID":"41f1fa06-a5b4-410b-8000-7b5ebbbf3a94","Type":"ContainerStarted","Data":"ea06be6ff41a1159ae1534cc1e7f6393f078c68df11055680e23724cd99f075e"}
Sep 09 06:51:30.219839 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.219815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal" event={"ID":"29fe8aee47d6454f6b21baaf9f280874","Type":"ContainerStarted","Data":"22f0754703ebcccb412868aaccca7541d33747d3e4df11ba608dc2a6bf168e8c"}
Sep 09 06:51:30.221467 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.221442    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b30aa0c-9473-4ad8-a676-e12441e128cf" containerID="afda8f53813dc06cb5f3d4b634a9e8050a758ff2b5659b5b4d38fd6a536db1b4" exitCode=0
Sep 09 06:51:30.221552 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.221528    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" event={"ID":"4b30aa0c-9473-4ad8-a676-e12441e128cf","Type":"ContainerDied","Data":"afda8f53813dc06cb5f3d4b634a9e8050a758ff2b5659b5b4d38fd6a536db1b4"}
Sep 09 06:51:30.225692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.225356    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"dbb627e521405d59315c99ee137adac3b8e3f7bda2c87a461b3ed15479a06f2a"}
Sep 09 06:51:30.225692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.225429    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"451ac23b34b5d4f74942696c0bd9ff3f37a2bc0c8a41a7fd5c22b56831d03d93"}
Sep 09 06:51:30.225692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.225472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"056d2634bbc9197ae63ade1d020996e1e6f07f819325a9000553ada17c84440c"}
Sep 09 06:51:30.225692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.225452    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rkmgz" podStartSLOduration=2.547321077 podStartE2EDuration="21.225436723s" totalImagesPullingTime="18.678115646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:30.22503508 +0000 UTC m=+22.595294168" watchObservedRunningTime="2026-09-09 06:51:30.225436723 +0000 UTC m=+22.595695812"
Sep 09 06:51:30.225692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.225521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"7eca9f6582ff7f6e7f32dfc86120ecaa644b84c9ad9eb47febb961b6b517ab9f"}
Sep 09 06:51:30.225692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.225566    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"32f63abef196795d993edccb43c5f1ff674b8f960ca468c8ceae247faac2efec"}
Sep 09 06:51:30.225692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.225636    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"0cbefea490ca41bcb13d18534afe7db15e1e9c3d9bc14ba68862eeb0d66be3c7"}
Sep 09 06:51:30.227075 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.227054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-rps4n" event={"ID":"7c31d1b6-7168-45b0-b317-e4bc606fedcf","Type":"ContainerStarted","Data":"4cbac5edc13adbb773fab753f32ea8aa0ec37f84eaf14b57f87cc23f36c00628"}
Sep 09 06:51:30.239856 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.239816    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-206.ec2.internal" podStartSLOduration=20.239803518 podStartE2EDuration="20.239803518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:30.239288876 +0000 UTC m=+22.609547967" watchObservedRunningTime="2026-09-09 06:51:30.239803518 +0000 UTC m=+22.610062606"
Sep 09 06:51:30.255820 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.255751    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-7dhhp" podStartSLOduration=2.505173 podStartE2EDuration="21.255733783s" totalImagesPullingTime="18.750560783s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:30.254741358 +0000 UTC m=+22.625000446" watchObservedRunningTime="2026-09-09 06:51:30.255733783 +0000 UTC m=+22.625992872"
Sep 09 06:51:30.287029 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.286981    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-6r2sr" podStartSLOduration=2.628870891 podStartE2EDuration="21.286966208s" totalImagesPullingTime="18.658095317s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:30.286786713 +0000 UTC m=+22.657045813" watchObservedRunningTime="2026-09-09 06:51:30.286966208 +0000 UTC m=+22.657225295"
Sep 09 06:51:30.287211 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.287177    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-97ml8" podStartSLOduration=2.60133051 podStartE2EDuration="21.287172645s" totalImagesPullingTime="18.685842135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:30.272557298 +0000 UTC m=+22.642816387" watchObservedRunningTime="2026-09-09 06:51:30.287172645 +0000 UTC m=+22.657431799"
Sep 09 06:51:30.617384 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:30.617353    2563 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 09 06:51:31.114067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.113967    2563 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-09T06:51:30.617381087Z","UUID":"557e9daf-0db6-4cd3-99a2-ac2a48b98081","Handler":null,"Name":"","Endpoint":""}
Sep 09 06:51:31.117189 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.117162    2563 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 09 06:51:31.117321 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.117200    2563 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 09 06:51:31.232053 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.232016    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd" event={"ID":"595a451a-b092-43db-91e4-ee3012869077","Type":"ContainerStarted","Data":"3521229d6a5325523e987a73430c78ccf4c20e314b367712f345414b412e5087"}
Sep 09 06:51:31.235871 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.235841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-62jhc" event={"ID":"b82c5253-36f3-4e09-8467-3085d29c0c65","Type":"ContainerStarted","Data":"10a70b7001c62c0db9bb617a91c550e295c4445d7d6011bd76171dc673f601be"}
Sep 09 06:51:31.250186 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.250137    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-62jhc" podStartSLOduration=3.560217603 podStartE2EDuration="22.250120083s" totalImagesPullingTime="18.68990248s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:31.250054077 +0000 UTC m=+23.620313166" watchObservedRunningTime="2026-09-09 06:51:31.250120083 +0000 UTC m=+23.620379184"
Sep 09 06:51:31.250498 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.250470    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-rps4n" podStartSLOduration=3.542706261 podStartE2EDuration="22.250459268s" totalImagesPullingTime="18.707753007s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:30.347935476 +0000 UTC m=+22.718194566" watchObservedRunningTime="2026-09-09 06:51:31.250459268 +0000 UTC m=+23.620718359"
Sep 09 06:51:31.588834 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.588799    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:31.589441 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:31.589418    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:32.110800 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:32.110766    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:32.110967 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:32.110809    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:32.110967 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:32.110772    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:32.110967 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:32.110906    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:32.111125 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:32.110958    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:32.111125 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:32.111051    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:32.240291 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:32.240253    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd" event={"ID":"595a451a-b092-43db-91e4-ee3012869077","Type":"ContainerStarted","Data":"1318a3d3d19f715cc87e1368d97748c9517913abb86561d154e46aff22608ab4"}
Sep 09 06:51:32.244270 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:32.244194    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"497b4d9aba81d052fb7fab3b8e111309c9e648aa81e76eb25a962dd97e2c231a"}
Sep 09 06:51:32.244479 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:32.244452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:32.245009 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:32.244990    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-97ml8"
Sep 09 06:51:32.257818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:32.257774    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-wwhmd" podStartSLOduration=2.556798659 podStartE2EDuration="23.257759473s" totalImagesPullingTime="20.700960814s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:32.257210957 +0000 UTC m=+24.627470055" watchObservedRunningTime="2026-09-09 06:51:32.257759473 +0000 UTC m=+24.628018811"
Sep 09 06:51:34.111114 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:34.110921    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:34.111574 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:34.110931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:34.111574 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:34.110937    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:34.111574 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:34.111206    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:34.111574 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:34.111330    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:34.111574 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:34.111422    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:35.254929 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:35.254901    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" event={"ID":"931199f9-eaba-4aec-9bf0-5ce95a9d37b0","Type":"ContainerStarted","Data":"fef1df64363a8c24e3c558dac2ae192ce5eef7186e70df7282146f14a7a243c0"}
Sep 09 06:51:35.256862 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:35.256736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:35.256862 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:35.256814    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:35.274012 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:35.273816    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:35.274155 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:35.274139    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:35.283859 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:35.283819    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt" podStartSLOduration=7.563101351 podStartE2EDuration="26.283805654s" totalImagesPullingTime="18.720704303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:35.282406099 +0000 UTC m=+27.652665189" watchObservedRunningTime="2026-09-09 06:51:35.283805654 +0000 UTC m=+27.654064742"
Sep 09 06:51:36.110637 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.110610    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:36.110845 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.110703    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:36.110845 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:36.110791    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:36.110845 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:36.110703    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:36.110845 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.110823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:36.110997 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:36.110900    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:36.262256 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.260410    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b30aa0c-9473-4ad8-a676-e12441e128cf" containerID="4f1073e84e0354222268001af4692a8b2f535dc30d16b46ce0ddc153aeb6db2d" exitCode=0
Sep 09 06:51:36.262256 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.260620    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 06:51:36.262256 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.262059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" event={"ID":"4b30aa0c-9473-4ad8-a676-e12441e128cf","Type":"ContainerDied","Data":"4f1073e84e0354222268001af4692a8b2f535dc30d16b46ce0ddc153aeb6db2d"}
Sep 09 06:51:36.858368 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.858330    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-984vs"]
Sep 09 06:51:36.858564 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.858478    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7px44"]
Sep 09 06:51:36.858659 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.858600    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:36.858784 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:36.858754    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:36.858910 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.858823    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:36.858984 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:36.858955    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:36.859858 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.859837    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dkrzk"]
Sep 09 06:51:36.859940 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:36.859931    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:36.860021 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:36.860002    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:37.263064 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:37.263036    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 06:51:38.267306 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:38.267275    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b30aa0c-9473-4ad8-a676-e12441e128cf" containerID="00591a034b91ba4be04c06d08aa3da1d6d9a2e6c2c0f619dde8aa1284e8e9d4b" exitCode=0
Sep 09 06:51:38.267686 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:38.267321    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" event={"ID":"4b30aa0c-9473-4ad8-a676-e12441e128cf","Type":"ContainerDied","Data":"00591a034b91ba4be04c06d08aa3da1d6d9a2e6c2c0f619dde8aa1284e8e9d4b"}
Sep 09 06:51:39.110873 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:39.110542    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:39.110954 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:39.110556    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:39.111015 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:39.110957    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:39.111067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:39.110568    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:39.111110 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:39.111055    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:39.111110 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:39.111106    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:39.273340 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:39.273260    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b30aa0c-9473-4ad8-a676-e12441e128cf" containerID="6c1198bbd2e195ffb5100893ad8d91158b0ec8eee70e7fc8cd2bbe338feeeb51" exitCode=0
Sep 09 06:51:39.273779 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:39.273355    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" event={"ID":"4b30aa0c-9473-4ad8-a676-e12441e128cf","Type":"ContainerDied","Data":"6c1198bbd2e195ffb5100893ad8d91158b0ec8eee70e7fc8cd2bbe338feeeb51"}
Sep 09 06:51:39.672944 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:39.672909    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:40.094372 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:40.094332    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:40.094635 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:40.094615    2563 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 09 06:51:40.106048 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:40.106023    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6xnwt"
Sep 09 06:51:40.137107 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:40.137077    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:40.439693 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:40.439654    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:40.506626 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:40.506579    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:40.639273 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:40.639243    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:40.979155 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:40.979123    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:41.080801 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:41.080768    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:41.110916 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:41.110882    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:41.111176 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:41.110928    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:41.111176 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:41.110985    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:41.111176 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:41.111025    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-984vs" podUID="2c131071-1759-46a1-9f59-d55efbbab153"
Sep 09 06:51:41.111176 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:41.111164    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-dkrzk" podUID="c5585130-4596-4fb1-8751-0dd1a062dc2e"
Sep 09 06:51:41.111424 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:41.111262    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7px44" podUID="857657b9-98d6-4f08-84cc-c832fa9d79ab"
Sep 09 06:51:41.814252 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:41.814218    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:41.814252 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:41.814256    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:41.814771 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:41.814373    2563 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:41.814771 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:41.814394    2563 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:41.814771 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:41.814438    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret podName:857657b9-98d6-4f08-84cc-c832fa9d79ab nodeName:}" failed. No retries permitted until 2026-09-09 06:52:13.814422563 +0000 UTC m=+66.184681629 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret") pod "global-pull-secret-syncer-7px44" (UID: "857657b9-98d6-4f08-84cc-c832fa9d79ab") : object "kube-system"/"original-pull-secret" not registered
Sep 09 06:51:41.814771 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:41.814452    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs podName:c5585130-4596-4fb1-8751-0dd1a062dc2e nodeName:}" failed. No retries permitted until 2026-09-09 06:52:13.814445765 +0000 UTC m=+66.184704831 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs") pod "network-metrics-daemon-dkrzk" (UID: "c5585130-4596-4fb1-8751-0dd1a062dc2e") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 09 06:51:41.979075 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:41.979040    2563 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-206.ec2.internal" event="NodeReady"
Sep 09 06:51:41.979255 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:41.979206    2563 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 09 06:51:42.016457 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.016383    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:42.016630 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:42.016572    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 09 06:51:42.016630 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:42.016617    2563 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 09 06:51:42.016630 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:42.016631    2563 projected.go:196] Error preparing data for projected volume kube-api-access-l25wb for pod openshift-network-diagnostics/network-check-target-984vs: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:42.016783 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:42.016692    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb podName:2c131071-1759-46a1-9f59-d55efbbab153 nodeName:}" failed. No retries permitted until 2026-09-09 06:52:14.016673666 +0000 UTC m=+66.386932738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-l25wb" (UniqueName: "kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb") pod "network-check-target-984vs" (UID: "2c131071-1759-46a1-9f59-d55efbbab153") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 09 06:51:42.026733 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.026708    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8n858"]
Sep 09 06:51:42.031979 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.031959    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wncxl"]
Sep 09 06:51:42.032128 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.032109    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.035419 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.035391    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 09 06:51:42.035523 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.035447    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 09 06:51:42.035523 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.035497    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n7krz\""
Sep 09 06:51:42.035663 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.035565    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wncxl"
Sep 09 06:51:42.037571 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.037543    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 09 06:51:42.037816 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.037777    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 09 06:51:42.037981 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.037966    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 09 06:51:42.038122 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.038101    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-dk92v\""
Sep 09 06:51:42.042642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.042620    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8n858"]
Sep 09 06:51:42.045213 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.045149    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wncxl"]
Sep 09 06:51:42.117578 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.117545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8ae03c8-633c-4fab-a8b3-6fb91116ee06-cert\") pod \"ingress-canary-wncxl\" (UID: \"d8ae03c8-633c-4fab-a8b3-6fb91116ee06\") " pod="openshift-ingress-canary/ingress-canary-wncxl"
Sep 09 06:51:42.117763 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.117606    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9262d9d7-02ad-4160-876d-b6fc06627736-config-volume\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.117763 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.117653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-562dt\" (UniqueName: \"kubernetes.io/projected/9262d9d7-02ad-4160-876d-b6fc06627736-kube-api-access-562dt\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.117763 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.117723    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9262d9d7-02ad-4160-876d-b6fc06627736-tmp-dir\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.117763 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.117756    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.117928 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.117807    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td4t4\" (UniqueName: \"kubernetes.io/projected/d8ae03c8-633c-4fab-a8b3-6fb91116ee06-kube-api-access-td4t4\") pod \"ingress-canary-wncxl\" (UID: \"d8ae03c8-633c-4fab-a8b3-6fb91116ee06\") " pod="openshift-ingress-canary/ingress-canary-wncxl"
Sep 09 06:51:42.218185 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.218156    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9262d9d7-02ad-4160-876d-b6fc06627736-tmp-dir\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.218352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.218205    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.218352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.218254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td4t4\" (UniqueName: \"kubernetes.io/projected/d8ae03c8-633c-4fab-a8b3-6fb91116ee06-kube-api-access-td4t4\") pod \"ingress-canary-wncxl\" (UID: \"d8ae03c8-633c-4fab-a8b3-6fb91116ee06\") " pod="openshift-ingress-canary/ingress-canary-wncxl"
Sep 09 06:51:42.218352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.218273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8ae03c8-633c-4fab-a8b3-6fb91116ee06-cert\") pod \"ingress-canary-wncxl\" (UID: \"d8ae03c8-633c-4fab-a8b3-6fb91116ee06\") " pod="openshift-ingress-canary/ingress-canary-wncxl"
Sep 09 06:51:42.218352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.218297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9262d9d7-02ad-4160-876d-b6fc06627736-config-volume\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.218352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.218325    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-562dt\" (UniqueName: \"kubernetes.io/projected/9262d9d7-02ad-4160-876d-b6fc06627736-kube-api-access-562dt\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.218626 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:42.218433    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 06:51:42.218626 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.218505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9262d9d7-02ad-4160-876d-b6fc06627736-tmp-dir\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.218626 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:42.218514    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls podName:9262d9d7-02ad-4160-876d-b6fc06627736 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:42.718493153 +0000 UTC m=+35.088752233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls") pod "dns-default-8n858" (UID: "9262d9d7-02ad-4160-876d-b6fc06627736") : secret "dns-default-metrics-tls" not found
Sep 09 06:51:42.219075 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.219049    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9262d9d7-02ad-4160-876d-b6fc06627736-config-volume\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.223211 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.223186    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d8ae03c8-633c-4fab-a8b3-6fb91116ee06-cert\") pod \"ingress-canary-wncxl\" (UID: \"d8ae03c8-633c-4fab-a8b3-6fb91116ee06\") " pod="openshift-ingress-canary/ingress-canary-wncxl"
Sep 09 06:51:42.229626 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.229578    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-562dt\" (UniqueName: \"kubernetes.io/projected/9262d9d7-02ad-4160-876d-b6fc06627736-kube-api-access-562dt\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.229732 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.229682    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td4t4\" (UniqueName: \"kubernetes.io/projected/d8ae03c8-633c-4fab-a8b3-6fb91116ee06-kube-api-access-td4t4\") pod \"ingress-canary-wncxl\" (UID: \"d8ae03c8-633c-4fab-a8b3-6fb91116ee06\") " pod="openshift-ingress-canary/ingress-canary-wncxl"
Sep 09 06:51:42.352745 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.352655    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wncxl"
Sep 09 06:51:42.722240 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:42.722200    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:42.722434 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:42.722353    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 06:51:42.722434 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:42.722415    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls podName:9262d9d7-02ad-4160-876d-b6fc06627736 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:43.722400007 +0000 UTC m=+36.092659078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls") pod "dns-default-8n858" (UID: "9262d9d7-02ad-4160-876d-b6fc06627736") : secret "dns-default-metrics-tls" not found
Sep 09 06:51:43.111217 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.110493    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:51:43.111217 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.110535    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:51:43.111217 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.110493    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:51:43.113573 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.113547    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 06:51:43.114516 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.114492    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 09 06:51:43.114749 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.114731    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 09 06:51:43.115001 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.114977    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgwvj\""
Sep 09 06:51:43.115189 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.115169    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 06:51:43.115383 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.115367    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wxxvd\""
Sep 09 06:51:43.567076 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.566903    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wncxl"]
Sep 09 06:51:43.588507 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:43.588470    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ae03c8_633c_4fab_a8b3_6fb91116ee06.slice/crio-0e746ed9422b072ab6e15b9a3ca6476407034a6e700d9ee4e7f8d289434143c9 WatchSource:0}: Error finding container 0e746ed9422b072ab6e15b9a3ca6476407034a6e700d9ee4e7f8d289434143c9: Status 404 returned error can't find the container with id 0e746ed9422b072ab6e15b9a3ca6476407034a6e700d9ee4e7f8d289434143c9
Sep 09 06:51:43.729355 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:43.729328    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:43.729504 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:43.729432    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 06:51:43.729504 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:43.729486    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls podName:9262d9d7-02ad-4160-876d-b6fc06627736 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:45.729473284 +0000 UTC m=+38.099732349 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls") pod "dns-default-8n858" (UID: "9262d9d7-02ad-4160-876d-b6fc06627736") : secret "dns-default-metrics-tls" not found
Sep 09 06:51:44.028741 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:44.028657    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:44.128291 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:44.128263    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:44.228961 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:44.228932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:44.288067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:44.287995    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b30aa0c-9473-4ad8-a676-e12441e128cf" containerID="b292156c933a01ff75de4ba3dcfbe7a37e3f5ce1bb81cd912b2d9df00685053b" exitCode=0
Sep 09 06:51:44.288222 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:44.288080    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" event={"ID":"4b30aa0c-9473-4ad8-a676-e12441e128cf","Type":"ContainerDied","Data":"b292156c933a01ff75de4ba3dcfbe7a37e3f5ce1bb81cd912b2d9df00685053b"}
Sep 09 06:51:44.289216 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:44.289198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wncxl" event={"ID":"d8ae03c8-633c-4fab-a8b3-6fb91116ee06","Type":"ContainerStarted","Data":"0e746ed9422b072ab6e15b9a3ca6476407034a6e700d9ee4e7f8d289434143c9"}
Sep 09 06:51:44.868942 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:44.868904    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:44.970085 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:44.970058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:45.295724 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:45.295685    2563 generic.go:356] "Generic (PLEG): container finished" podID="4b30aa0c-9473-4ad8-a676-e12441e128cf" containerID="db3e7221d3af04c05ff5b8e3f0f502ce9ea850a1eaa4700d65b1e8e8dcaf5e86" exitCode=0
Sep 09 06:51:45.296166 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:45.295766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" event={"ID":"4b30aa0c-9473-4ad8-a676-e12441e128cf","Type":"ContainerDied","Data":"db3e7221d3af04c05ff5b8e3f0f502ce9ea850a1eaa4700d65b1e8e8dcaf5e86"}
Sep 09 06:51:45.745850 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:45.745608    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:45.746018 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:45.745761    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 06:51:45.746018 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:45.745941    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls podName:9262d9d7-02ad-4160-876d-b6fc06627736 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:49.745919946 +0000 UTC m=+42.116179032 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls") pod "dns-default-8n858" (UID: "9262d9d7-02ad-4160-876d-b6fc06627736") : secret "dns-default-metrics-tls" not found
Sep 09 06:51:46.302285 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:46.302243    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" event={"ID":"4b30aa0c-9473-4ad8-a676-e12441e128cf","Type":"ContainerStarted","Data":"30281afa565c7a325382fb0d0b6f18da50aa81d50966d81257c6b17dc73e2707"}
Sep 09 06:51:46.323739 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:46.323694    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-kzxzm" podStartSLOduration=7.685046131 podStartE2EDuration="37.323641743s" totalImagesPullingTime="29.638595612s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:46.322388964 +0000 UTC m=+38.692648053" watchObservedRunningTime="2026-09-09 06:51:46.323641743 +0000 UTC m=+38.693900830"
Sep 09 06:51:46.444514 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:46.444469    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:46.510092 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:46.510056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:47.305742 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:47.305649    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wncxl" event={"ID":"d8ae03c8-633c-4fab-a8b3-6fb91116ee06","Type":"ContainerStarted","Data":"b0df44ee1ac769b30e082b9d499fee571962067cc6762235c9e7e8efcaff1097"}
Sep 09 06:51:47.322401 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:47.322350    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wncxl" podStartSLOduration=1.971142486 podStartE2EDuration="5.322334701s" totalImagesPullingTime="3.351192215s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:47.32119848 +0000 UTC m=+39.691457582" watchObservedRunningTime="2026-09-09 06:51:47.322334701 +0000 UTC m=+39.692593789"
Sep 09 06:51:47.654463 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:47.654388    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:47.756305 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:47.756270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:49.774559 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:49.774521    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:49.775017 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:49.774654    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 06:51:49.775017 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:49.774709    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls podName:9262d9d7-02ad-4160-876d-b6fc06627736 nodeName:}" failed. No retries permitted until 2026-09-09 06:51:57.774695083 +0000 UTC m=+50.144954153 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls") pod "dns-default-8n858" (UID: "9262d9d7-02ad-4160-876d-b6fc06627736") : secret "dns-default-metrics-tls" not found
Sep 09 06:51:53.367759 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.367686    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-58674b4548-wzsrc"]
Sep 09 06:51:53.388775 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.388747    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l26h6"]
Sep 09 06:51:53.388930 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.388891    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.391193 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.391174    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 09 06:51:53.391426 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.391409    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 09 06:51:53.391545 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.391417    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wmx4b\""
Sep 09 06:51:53.392141 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.392122    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 09 06:51:53.392226 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.392153    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 09 06:51:53.392382 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.392363    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 09 06:51:53.392480 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.392399    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 09 06:51:53.405755 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.405733    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l26h6"]
Sep 09 06:51:53.405755 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.405764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58674b4548-wzsrc"]
Sep 09 06:51:53.405940 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.405872    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-l26h6"
Sep 09 06:51:53.408903 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.408887    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-r4djn\""
Sep 09 06:51:53.497250 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.497209    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-stats-auth\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.497415 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.497264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ls28\" (UniqueName: \"kubernetes.io/projected/af681382-11e8-4b66-8f2e-31e7646c5c34-kube-api-access-2ls28\") pod \"network-check-source-685456655f-l26h6\" (UID: \"af681382-11e8-4b66-8f2e-31e7646c5c34\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l26h6"
Sep 09 06:51:53.497415 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.497294    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-default-certificate\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.497415 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.497310    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hlp6b\" (UniqueName: \"kubernetes.io/projected/e666318d-cdcd-483c-b61f-638bccc6cacf-kube-api-access-hlp6b\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.497415 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.497326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.497415 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.497376    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.527193 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.527163    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:53.559161 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.559133    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"]
Sep 09 06:51:53.576561 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.576536    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-85b454874b-zgbdk"]
Sep 09 06:51:53.576687 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.576657    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.580685 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.580667    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 09 06:51:53.580946 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.580931    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 09 06:51:53.581019 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.580936    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 09 06:51:53.581462 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.581446    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 09 06:51:53.581462 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.581452    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-nbplx\""
Sep 09 06:51:53.597806 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.597786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-stats-auth\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.597933 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.597843    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ls28\" (UniqueName: \"kubernetes.io/projected/af681382-11e8-4b66-8f2e-31e7646c5c34-kube-api-access-2ls28\") pod \"network-check-source-685456655f-l26h6\" (UID: \"af681382-11e8-4b66-8f2e-31e7646c5c34\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l26h6"
Sep 09 06:51:53.597933 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.597877    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-default-certificate\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.597933 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.597902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hlp6b\" (UniqueName: \"kubernetes.io/projected/e666318d-cdcd-483c-b61f-638bccc6cacf-kube-api-access-hlp6b\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.597933 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.597926    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.598128 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.597978    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.598128 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:53.598086    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 06:51:53.598205 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:53.598139    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:51:54.098121807 +0000 UTC m=+46.468380876 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : configmap references non-existent config key: service-ca.crt
Sep 09 06:51:53.598205 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:53.598159    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:51:54.098152556 +0000 UTC m=+46.468411623 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : secret "router-metrics-certs-default" not found
Sep 09 06:51:53.601792 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.601774    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-default-certificate\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.601932 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.601775    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-stats-auth\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.603014 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.602992    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"]
Sep 09 06:51:53.603014 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.603023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-zgbdk"]
Sep 09 06:51:53.603148 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.603128    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.605936 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.605918    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Sep 09 06:51:53.606070 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.605919    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Sep 09 06:51:53.606070 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.606030    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-mbc4z\""
Sep 09 06:51:53.606314 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.606170    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Sep 09 06:51:53.606415 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.606397    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Sep 09 06:51:53.609352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.609334    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hlp6b\" (UniqueName: \"kubernetes.io/projected/e666318d-cdcd-483c-b61f-638bccc6cacf-kube-api-access-hlp6b\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:53.612017 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.612000    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ls28\" (UniqueName: \"kubernetes.io/projected/af681382-11e8-4b66-8f2e-31e7646c5c34-kube-api-access-2ls28\") pod \"network-check-source-685456655f-l26h6\" (UID: \"af681382-11e8-4b66-8f2e-31e7646c5c34\") " pod="openshift-network-diagnostics/network-check-source-685456655f-l26h6"
Sep 09 06:51:53.612338 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.612325    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Sep 09 06:51:53.613328 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.613307    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:53.698535 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.698499    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1881221-61f9-4b88-8296-96fc2078afa5-serving-cert\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.698693 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.698545    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2a5575d9-f5a0-4de3-9833-f30eed7135cb-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.698693 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.698573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxd46\" (UniqueName: \"kubernetes.io/projected/2a5575d9-f5a0-4de3-9833-f30eed7135cb-kube-api-access-rxd46\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.698693 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.698619    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1881221-61f9-4b88-8296-96fc2078afa5-config\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.698693 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.698642    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1881221-61f9-4b88-8296-96fc2078afa5-trusted-ca\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.698887 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.698729    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nffwq\" (UniqueName: \"kubernetes.io/projected/b1881221-61f9-4b88-8296-96fc2078afa5-kube-api-access-nffwq\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.698887 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.698768    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a5575d9-f5a0-4de3-9833-f30eed7135cb-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.708985 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.708960    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:53.714258 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.714234    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-685456655f-l26h6"
Sep 09 06:51:53.800096 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.800066    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nffwq\" (UniqueName: \"kubernetes.io/projected/b1881221-61f9-4b88-8296-96fc2078afa5-kube-api-access-nffwq\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.800204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.800110    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a5575d9-f5a0-4de3-9833-f30eed7135cb-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.800204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.800152    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1881221-61f9-4b88-8296-96fc2078afa5-serving-cert\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.800204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.800173    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2a5575d9-f5a0-4de3-9833-f30eed7135cb-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.800204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.800188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxd46\" (UniqueName: \"kubernetes.io/projected/2a5575d9-f5a0-4de3-9833-f30eed7135cb-kube-api-access-rxd46\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.800204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.800204    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1881221-61f9-4b88-8296-96fc2078afa5-config\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.800438 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.800254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1881221-61f9-4b88-8296-96fc2078afa5-trusted-ca\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.800853 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.800833    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2a5575d9-f5a0-4de3-9833-f30eed7135cb-config\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.801186 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.801164    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/b1881221-61f9-4b88-8296-96fc2078afa5-config\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.801249 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.801235    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/b1881221-61f9-4b88-8296-96fc2078afa5-trusted-ca\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.802634 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.802603    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/b1881221-61f9-4b88-8296-96fc2078afa5-serving-cert\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.802909 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.802891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2a5575d9-f5a0-4de3-9833-f30eed7135cb-serving-cert\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.808031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.808008    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxd46\" (UniqueName: \"kubernetes.io/projected/2a5575d9-f5a0-4de3-9833-f30eed7135cb-kube-api-access-rxd46\") pod \"kube-storage-version-migrator-operator-8d7dd4675-pfbvh\" (UID: \"2a5575d9-f5a0-4de3-9833-f30eed7135cb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.808120 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.808060    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nffwq\" (UniqueName: \"kubernetes.io/projected/b1881221-61f9-4b88-8296-96fc2078afa5-kube-api-access-nffwq\") pod \"console-operator-85b454874b-zgbdk\" (UID: \"b1881221-61f9-4b88-8296-96fc2078afa5\") " pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:53.884721 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.884656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"
Sep 09 06:51:53.925345 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:53.925322    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:51:54.002549 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:54.001764    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-685456655f-l26h6"]
Sep 09 06:51:54.005497 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:54.005469    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf681382_11e8_4b66_8f2e_31e7646c5c34.slice/crio-a1a2a70c2d44fa0fddddb7222d07b008923c8b075562b1577308b77e45ae9510 WatchSource:0}: Error finding container a1a2a70c2d44fa0fddddb7222d07b008923c8b075562b1577308b77e45ae9510: Status 404 returned error can't find the container with id a1a2a70c2d44fa0fddddb7222d07b008923c8b075562b1577308b77e45ae9510
Sep 09 06:51:54.103445 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:54.103422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:54.103613 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:54.103494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:54.103613 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:54.103573    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 06:51:54.103694 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:54.103621    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:51:55.103605584 +0000 UTC m=+47.473864663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : configmap references non-existent config key: service-ca.crt
Sep 09 06:51:54.103694 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:54.103656    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:51:55.103641662 +0000 UTC m=+47.473900729 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : secret "router-metrics-certs-default" not found
Sep 09 06:51:54.182009 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:54.181983    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh"]
Sep 09 06:51:54.184074 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:54.184051    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a5575d9_f5a0_4de3_9833_f30eed7135cb.slice/crio-2223ad0188746bcf701419b2d356fa0e99a83f72272256b99dece3d01ab1684f WatchSource:0}: Error finding container 2223ad0188746bcf701419b2d356fa0e99a83f72272256b99dece3d01ab1684f: Status 404 returned error can't find the container with id 2223ad0188746bcf701419b2d356fa0e99a83f72272256b99dece3d01ab1684f
Sep 09 06:51:54.215294 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:54.215271    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-85b454874b-zgbdk"]
Sep 09 06:51:54.217958 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:51:54.217936    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1881221_61f9_4b88_8296_96fc2078afa5.slice/crio-ff40ff02394903fb9787872f01566992039820a29ff7bc35c3dfbb414d4844af WatchSource:0}: Error finding container ff40ff02394903fb9787872f01566992039820a29ff7bc35c3dfbb414d4844af: Status 404 returned error can't find the container with id ff40ff02394903fb9787872f01566992039820a29ff7bc35c3dfbb414d4844af
Sep 09 06:51:54.323652 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:54.323611    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" event={"ID":"b1881221-61f9-4b88-8296-96fc2078afa5","Type":"ContainerStarted","Data":"ff40ff02394903fb9787872f01566992039820a29ff7bc35c3dfbb414d4844af"}
Sep 09 06:51:54.324709 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:54.324685    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh" event={"ID":"2a5575d9-f5a0-4de3-9833-f30eed7135cb","Type":"ContainerStarted","Data":"2223ad0188746bcf701419b2d356fa0e99a83f72272256b99dece3d01ab1684f"}
Sep 09 06:51:54.325784 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:54.325766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-l26h6" event={"ID":"af681382-11e8-4b66-8f2e-31e7646c5c34","Type":"ContainerStarted","Data":"a1a2a70c2d44fa0fddddb7222d07b008923c8b075562b1577308b77e45ae9510"}
Sep 09 06:51:55.111238 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:55.111195    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:55.111716 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:55.111321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:55.111716 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:55.111350    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 06:51:55.111716 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:55.111420    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:51:57.111399778 +0000 UTC m=+49.481658866 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : secret "router-metrics-certs-default" not found
Sep 09 06:51:55.111716 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:55.111438    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:51:57.111429925 +0000 UTC m=+49.481688990 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : configmap references non-existent config key: service-ca.crt
Sep 09 06:51:56.575233 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:56.575187    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:56.631635 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:56.631595    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:57.128439 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:57.128360    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:57.128439 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:57.128438    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:51:57.128713 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:57.128566    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:52:01.128541367 +0000 UTC m=+53.498800436 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : configmap references non-existent config key: service-ca.crt
Sep 09 06:51:57.128713 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:57.128623    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 06:51:57.128713 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:57.128688    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:52:01.128675295 +0000 UTC m=+53.498934361 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : secret "router-metrics-certs-default" not found
Sep 09 06:51:57.809796 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:57.809760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:57.834201 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:57.834167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:51:57.834389 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:57.834341    2563 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 09 06:51:57.834440 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:51:57.834417    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls podName:9262d9d7-02ad-4160-876d-b6fc06627736 nodeName:}" failed. No retries permitted until 2026-09-09 06:52:13.834396079 +0000 UTC m=+66.204655163 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls") pod "dns-default-8n858" (UID: "9262d9d7-02ad-4160-876d-b6fc06627736") : secret "dns-default-metrics-tls" not found
Sep 09 06:51:57.910807 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:57.910746    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:59.341047 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.340953    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-685456655f-l26h6" event={"ID":"af681382-11e8-4b66-8f2e-31e7646c5c34","Type":"ContainerStarted","Data":"acf5cb45a53e2adb79948efef07575fc9193deb43ec71e4221c40632df2d81a3"}
Sep 09 06:51:59.342699 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.342679    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/0.log"
Sep 09 06:51:59.342816 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.342713    2563 generic.go:356] "Generic (PLEG): container finished" podID="b1881221-61f9-4b88-8296-96fc2078afa5" containerID="e8236e6033064f0bc2f1e0180c98b112e899bd00b54fb6a87a716d6e75be60e3" exitCode=255
Sep 09 06:51:59.342816 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.342794    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" event={"ID":"b1881221-61f9-4b88-8296-96fc2078afa5","Type":"ContainerDied","Data":"e8236e6033064f0bc2f1e0180c98b112e899bd00b54fb6a87a716d6e75be60e3"}
Sep 09 06:51:59.342970 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.342952    2563 scope.go:117] "RemoveContainer" containerID="e8236e6033064f0bc2f1e0180c98b112e899bd00b54fb6a87a716d6e75be60e3"
Sep 09 06:51:59.344476 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.344443    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh" event={"ID":"2a5575d9-f5a0-4de3-9833-f30eed7135cb","Type":"ContainerStarted","Data":"8116201e557d55f4e18d57ff938b9dad99a612d5203975d9b11fe4e3e14623b3"}
Sep 09 06:51:59.357654 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.357609    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-685456655f-l26h6" podStartSLOduration=1.414697111 podStartE2EDuration="6.357576794s" totalImagesPullingTime="4.942879683s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:59.356325597 +0000 UTC m=+51.726584686" watchObservedRunningTime="2026-09-09 06:51:59.357576794 +0000 UTC m=+51.727835883"
Sep 09 06:51:59.392373 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.392332    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh" podStartSLOduration=1.637643682 podStartE2EDuration="6.39231373s" totalImagesPullingTime="4.754670048s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:51:59.390993702 +0000 UTC m=+51.761252791" watchObservedRunningTime="2026-09-09 06:51:59.39231373 +0000 UTC m=+51.762572818"
Sep 09 06:51:59.645522 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.645446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:51:59.873408 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.873370    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zlcml"]
Sep 09 06:51:59.888947 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.888913    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:51:59.889821 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.889795    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zlcml"]
Sep 09 06:51:59.891571 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.891500    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 09 06:51:59.891707 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.891502    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nqhdr\""
Sep 09 06:51:59.891785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.891733    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 09 06:51:59.892692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.892673    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 09 06:51:59.892894 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:51:59.892870    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 09 06:52:00.051814 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.051766    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjhgk\" (UniqueName: \"kubernetes.io/projected/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-api-access-xjhgk\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.051992 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.051844    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.051992 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.051874    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.051992 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.051917    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f96657d-2047-4456-94fb-5e5fa0d9e3db-data-volume\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.051992 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.051965    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f96657d-2047-4456-94fb-5e5fa0d9e3db-crio-socket\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.123228 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.123197    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:00.152576 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.152493    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjhgk\" (UniqueName: \"kubernetes.io/projected/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-api-access-xjhgk\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.152576 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.152538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.152576 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.152556    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.152847 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.152609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f96657d-2047-4456-94fb-5e5fa0d9e3db-data-volume\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.152847 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.152658    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 06:52:00.152847 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.152684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f96657d-2047-4456-94fb-5e5fa0d9e3db-crio-socket\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.152847 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.152700    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:00.152847 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.152733    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:00.652715471 +0000 UTC m=+53.022974541 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : configmap "kube-rbac-proxy" not found
Sep 09 06:52:00.152847 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.152753    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:00.652747045 +0000 UTC m=+53.023006111 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:00.153110 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.152882    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/6f96657d-2047-4456-94fb-5e5fa0d9e3db-crio-socket\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.153110 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.153047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/6f96657d-2047-4456-94fb-5e5fa0d9e3db-data-volume\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.160914 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.160890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjhgk\" (UniqueName: \"kubernetes.io/projected/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-api-access-xjhgk\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.223272 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.223238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:00.348656 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.348613    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:00.349458 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.349441    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 06:52:00.349867 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.349853    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/0.log"
Sep 09 06:52:00.349930 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.349886    2563 generic.go:356] "Generic (PLEG): container finished" podID="b1881221-61f9-4b88-8296-96fc2078afa5" containerID="609fa502e96049e29f4a4f3061eeab35c21b8ad41b83a1747f8fc5163532ba46" exitCode=255
Sep 09 06:52:00.349930 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.349914    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" event={"ID":"b1881221-61f9-4b88-8296-96fc2078afa5","Type":"ContainerDied","Data":"609fa502e96049e29f4a4f3061eeab35c21b8ad41b83a1747f8fc5163532ba46"}
Sep 09 06:52:00.350016 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.349975    2563 scope.go:117] "RemoveContainer" containerID="e8236e6033064f0bc2f1e0180c98b112e899bd00b54fb6a87a716d6e75be60e3"
Sep 09 06:52:00.350235 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.350213    2563 scope.go:117] "RemoveContainer" containerID="609fa502e96049e29f4a4f3061eeab35c21b8ad41b83a1747f8fc5163532ba46"
Sep 09 06:52:00.350410 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.350388    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-zgbdk_openshift-console-operator(b1881221-61f9-4b88-8296-96fc2078afa5)\"" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" podUID="b1881221-61f9-4b88-8296-96fc2078afa5"
Sep 09 06:52:00.409791 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.409715    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:00.657256 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.657217    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.657256 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.657254    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:00.657485 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.657347    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 06:52:00.657485 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.657354    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:00.657485 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.657441    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:01.657418232 +0000 UTC m=+54.027677311 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : configmap "kube-rbac-proxy" not found
Sep 09 06:52:00.657485 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:00.657487    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:01.657474449 +0000 UTC m=+54.027733514 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:00.685927 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.685896    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:00.688906 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.688880    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4"]
Sep 09 06:52:00.691413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.691398    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4"
Sep 09 06:52:00.693639 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.693621    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 09 06:52:00.693920 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.693904    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-qg2cg\""
Sep 09 06:52:00.693973 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.693919    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 09 06:52:00.702455 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.702435    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4"]
Sep 09 06:52:00.757955 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.757918    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x25bv\" (UniqueName: \"kubernetes.io/projected/26903f73-be59-4b3a-9052-6218bddeacb2-kube-api-access-x25bv\") pod \"migrator-fdbb8fdff-qspv4\" (UID: \"26903f73-be59-4b3a-9052-6218bddeacb2\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4"
Sep 09 06:52:00.782186 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.782144    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:00.859241 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.859209    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x25bv\" (UniqueName: \"kubernetes.io/projected/26903f73-be59-4b3a-9052-6218bddeacb2-kube-api-access-x25bv\") pod \"migrator-fdbb8fdff-qspv4\" (UID: \"26903f73-be59-4b3a-9052-6218bddeacb2\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4"
Sep 09 06:52:00.867364 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.867344    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x25bv\" (UniqueName: \"kubernetes.io/projected/26903f73-be59-4b3a-9052-6218bddeacb2-kube-api-access-x25bv\") pod \"migrator-fdbb8fdff-qspv4\" (UID: \"26903f73-be59-4b3a-9052-6218bddeacb2\") " pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4"
Sep 09 06:52:00.883189 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:00.883164    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:01.000695 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.000614    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4"
Sep 09 06:52:01.162079 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.162041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:01.162249 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.162134    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:01.162249 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:01.162192    2563 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 09 06:52:01.162324 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:01.162259    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:52:09.162242279 +0000 UTC m=+61.532501350 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : secret "router-metrics-certs-default" not found
Sep 09 06:52:01.162324 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:01.162274    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:52:09.162268229 +0000 UTC m=+61.532527295 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : configmap references non-existent config key: service-ca.crt
Sep 09 06:52:01.266363 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.266287    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4"]
Sep 09 06:52:01.269867 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:01.269839    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26903f73_be59_4b3a_9052_6218bddeacb2.slice/crio-2ea6bc894f1716a8fd07bda5a9983c5e455923538b57bf090d6579ca8f31aaec WatchSource:0}: Error finding container 2ea6bc894f1716a8fd07bda5a9983c5e455923538b57bf090d6579ca8f31aaec: Status 404 returned error can't find the container with id 2ea6bc894f1716a8fd07bda5a9983c5e455923538b57bf090d6579ca8f31aaec
Sep 09 06:52:01.354244 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.354216    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 06:52:01.354651 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.354576    2563 scope.go:117] "RemoveContainer" containerID="609fa502e96049e29f4a4f3061eeab35c21b8ad41b83a1747f8fc5163532ba46"
Sep 09 06:52:01.354781 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:01.354760    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-zgbdk_openshift-console-operator(b1881221-61f9-4b88-8296-96fc2078afa5)\"" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" podUID="b1881221-61f9-4b88-8296-96fc2078afa5"
Sep 09 06:52:01.355494 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.355473    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4" event={"ID":"26903f73-be59-4b3a-9052-6218bddeacb2","Type":"ContainerStarted","Data":"2ea6bc894f1716a8fd07bda5a9983c5e455923538b57bf090d6579ca8f31aaec"}
Sep 09 06:52:01.664510 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.664418    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:01.664510 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:01.664457    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:01.664722 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:01.664533    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 06:52:01.664722 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:01.664580    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:01.664722 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:01.664630    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:03.664612233 +0000 UTC m=+56.034871305 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : configmap "kube-rbac-proxy" not found
Sep 09 06:52:01.664722 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:01.664644    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:03.664638402 +0000 UTC m=+56.034897468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:02.000835 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:02.000802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:02.019346 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:02.019318    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkmgz_11d5f98d-2bb0-4703-9b3a-2d72d8690e35/dns-node-resolver/0.log"
Sep 09 06:52:02.052866 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:02.052826    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:02.620243 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:02.620220    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6r2sr_635184e6-443f-4df2-99e0-496ddc9fbe02/node-ca/0.log"
Sep 09 06:52:02.904104 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:02.904028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:03.363833 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.363799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4" event={"ID":"26903f73-be59-4b3a-9052-6218bddeacb2","Type":"ContainerStarted","Data":"206d97762d3fa1dda924a203387da5e4f9d6e723fc8f0b7844937af49ddfd13b"}
Sep 09 06:52:03.364000 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.363853    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4" event={"ID":"26903f73-be59-4b3a-9052-6218bddeacb2","Type":"ContainerStarted","Data":"df43d6652c5474c11d3cf4a6a6175cc4596d72f6207542030a55a5afab986f0a"}
Sep 09 06:52:03.387277 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.387221    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-fdbb8fdff-qspv4" podStartSLOduration=2.164531343 podStartE2EDuration="3.387204868s" totalImagesPullingTime="1.222673525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:03.386045085 +0000 UTC m=+55.756304188" watchObservedRunningTime="2026-09-09 06:52:03.387204868 +0000 UTC m=+55.757463955"
Sep 09 06:52:03.680887 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.680837    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:03.680887 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.680893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:03.681293 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:03.680957    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 06:52:03.681293 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:03.681042    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:07.681025031 +0000 UTC m=+60.051284113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : configmap "kube-rbac-proxy" not found
Sep 09 06:52:03.681293 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:03.681069    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:03.681293 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:03.681134    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:07.681121318 +0000 UTC m=+60.051380385 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:03.722009 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.721979    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:03.819669 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.819643    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wncxl_d8ae03c8-633c-4fab-a8b3-6fb91116ee06/serve-healthcheck-canary/0.log"
Sep 09 06:52:03.925447 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.925393    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:52:03.925789 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.925492    2563 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:52:03.925873 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:03.925860    2563 scope.go:117] "RemoveContainer" containerID="609fa502e96049e29f4a4f3061eeab35c21b8ad41b83a1747f8fc5163532ba46"
Sep 09 06:52:03.926022 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:03.926004    2563 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-85b454874b-zgbdk_openshift-console-operator(b1881221-61f9-4b88-8296-96fc2078afa5)\"" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" podUID="b1881221-61f9-4b88-8296-96fc2078afa5"
Sep 09 06:52:04.150537 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:04.150448    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:04.260561 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:04.260520    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:04.358247 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:04.358214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:04.729597 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:04.729555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:05.119480 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:05.119393    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:05.925019 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:05.924985    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:06.125472 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:06.125440    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:06.438870 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:06.438844    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:06.538982 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:06.538949    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:06.726559 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:06.726489    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:07.132112 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:07.132037    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:07.434939 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:07.434915    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:07.533457 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:07.533425    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:07.633529 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:07.633496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:07.713925 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:07.713859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:07.713925 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:07.713898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:07.714044 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:07.713980    2563 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 09 06:52:07.714044 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:07.713999    2563 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:07.714044 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:07.714044    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:15.714025945 +0000 UTC m=+68.084285015 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : configmap "kube-rbac-proxy" not found
Sep 09 06:52:07.714139 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:07.714059    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls podName:6f96657d-2047-4456-94fb-5e5fa0d9e3db nodeName:}" failed. No retries permitted until 2026-09-09 06:52:15.714052293 +0000 UTC m=+68.084311359 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zlcml" (UID: "6f96657d-2047-4456-94fb-5e5fa0d9e3db") : secret "insights-runtime-extractor-tls" not found
Sep 09 06:52:08.129726 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:08.129651    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:08.231618 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:08.231574    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:08.331225 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:08.331191    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:08.527278 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:08.527246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:09.224643 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:09.224575    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:09.224912 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:09.224669    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:09.224912 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:09.224744    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle podName:e666318d-cdcd-483c-b61f-638bccc6cacf nodeName:}" failed. No retries permitted until 2026-09-09 06:52:25.224721445 +0000 UTC m=+77.594980528 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle") pod "router-default-58674b4548-wzsrc" (UID: "e666318d-cdcd-483c-b61f-638bccc6cacf") : configmap references non-existent config key: service-ca.crt
Sep 09 06:52:09.226961 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:09.226939    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e666318d-cdcd-483c-b61f-638bccc6cacf-metrics-certs\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:09.721740 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:09.721700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:09.919666 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:09.919632    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:10.019818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:10.019745    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:10.129456 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:10.129430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:10.325325 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:10.325244    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:10.523445 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:10.523414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:11.555181 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:11.555146    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:11.720263 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:11.720236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:12.117942 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:12.117908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:12.323239 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:12.323210    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:12.522689 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:12.522658    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:13.011168 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.011136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:13.525802 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.525770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:13.745916 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.745885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:13.865999 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.865922    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:52:13.865999 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.865959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:52:13.866171 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.866064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:52:13.868250 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.868232    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9262d9d7-02ad-4160-876d-b6fc06627736-metrics-tls\") pod \"dns-default-8n858\" (UID: \"9262d9d7-02ad-4160-876d-b6fc06627736\") " pod="openshift-dns/dns-default-8n858"
Sep 09 06:52:13.868648 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.868630    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 09 06:52:13.868691 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.868675    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 09 06:52:13.878748 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.878724    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c5585130-4596-4fb1-8751-0dd1a062dc2e-metrics-certs\") pod \"network-metrics-daemon-dkrzk\" (UID: \"c5585130-4596-4fb1-8751-0dd1a062dc2e\") " pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:52:13.878748 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.878740    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/857657b9-98d6-4f08-84cc-c832fa9d79ab-original-pull-secret\") pod \"global-pull-secret-syncer-7px44\" (UID: \"857657b9-98d6-4f08-84cc-c832fa9d79ab\") " pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:52:13.929147 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:13.929117    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:14.030653 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.030622    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7px44"
Sep 09 06:52:14.038410 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.038389    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-bgwvj\""
Sep 09 06:52:14.046614 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.046576    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-dkrzk"
Sep 09 06:52:14.067777 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.067746    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:52:14.071161 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.071132    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l25wb\" (UniqueName: \"kubernetes.io/projected/2c131071-1759-46a1-9f59-d55efbbab153-kube-api-access-l25wb\") pod \"network-check-target-984vs\" (UID: \"2c131071-1759-46a1-9f59-d55efbbab153\") " pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:52:14.076287 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.076267    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:14.148139 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.148115    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-n7krz\""
Sep 09 06:52:14.156369 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.156300    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8n858"
Sep 09 06:52:14.178442 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.178420    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:14.276894 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.276864    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:14.326111 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.326082    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-wxxvd\""
Sep 09 06:52:14.334056 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.334036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:52:14.336402 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.336308    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7px44"]
Sep 09 06:52:14.338956 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:14.338933    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857657b9_98d6_4f08_84cc_c832fa9d79ab.slice/crio-4324f1932277a1099d77c9f153fa6ab45bd2f7eb4c664b74bae21d3c3d20e92b WatchSource:0}: Error finding container 4324f1932277a1099d77c9f153fa6ab45bd2f7eb4c664b74bae21d3c3d20e92b: Status 404 returned error can't find the container with id 4324f1932277a1099d77c9f153fa6ab45bd2f7eb4c664b74bae21d3c3d20e92b
Sep 09 06:52:14.366337 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.366314    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-dkrzk"]
Sep 09 06:52:14.369758 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:14.369733    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5585130_4596_4fb1_8751_0dd1a062dc2e.slice/crio-74baa59321de6e8e67875020f2ce180d35f4dc5c5ef4e32d471e8a7e8a43f00f WatchSource:0}: Error finding container 74baa59321de6e8e67875020f2ce180d35f4dc5c5ef4e32d471e8a7e8a43f00f: Status 404 returned error can't find the container with id 74baa59321de6e8e67875020f2ce180d35f4dc5c5ef4e32d471e8a7e8a43f00f
Sep 09 06:52:14.396574 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.396543    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dkrzk" event={"ID":"c5585130-4596-4fb1-8751-0dd1a062dc2e","Type":"ContainerStarted","Data":"74baa59321de6e8e67875020f2ce180d35f4dc5c5ef4e32d471e8a7e8a43f00f"}
Sep 09 06:52:14.397692 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.397661    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7px44" event={"ID":"857657b9-98d6-4f08-84cc-c832fa9d79ab","Type":"ContainerStarted","Data":"4324f1932277a1099d77c9f153fa6ab45bd2f7eb4c664b74bae21d3c3d20e92b"}
Sep 09 06:52:14.453044 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.453015    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8n858"]
Sep 09 06:52:14.456120 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:14.456087    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9262d9d7_02ad_4160_876d_b6fc06627736.slice/crio-05bb2fd0e6365861fa9c4a16d2c10115003931e708fd9dd1137a259b2c4f6cdf WatchSource:0}: Error finding container 05bb2fd0e6365861fa9c4a16d2c10115003931e708fd9dd1137a259b2c4f6cdf: Status 404 returned error can't find the container with id 05bb2fd0e6365861fa9c4a16d2c10115003931e708fd9dd1137a259b2c4f6cdf
Sep 09 06:52:14.523115 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.523081    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:14.625829 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:14.625756    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-984vs"]
Sep 09 06:52:14.628862 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:14.628838    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c131071_1759_46a1_9f59_d55efbbab153.slice/crio-611e341e8e6d79e788e891c4c56594bfeb30cf265390f12d0e82ed31a5e417d4 WatchSource:0}: Error finding container 611e341e8e6d79e788e891c4c56594bfeb30cf265390f12d0e82ed31a5e417d4: Status 404 returned error can't find the container with id 611e341e8e6d79e788e891c4c56594bfeb30cf265390f12d0e82ed31a5e417d4
Sep 09 06:52:15.405486 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.405444    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-984vs" event={"ID":"2c131071-1759-46a1-9f59-d55efbbab153","Type":"ContainerStarted","Data":"d47a161d074725d563bd14aba0e949c7fd90525d6e53c1b5c576fa5925cc94c2"}
Sep 09 06:52:15.405956 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.405509    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-984vs" event={"ID":"2c131071-1759-46a1-9f59-d55efbbab153","Type":"ContainerStarted","Data":"611e341e8e6d79e788e891c4c56594bfeb30cf265390f12d0e82ed31a5e417d4"}
Sep 09 06:52:15.405956 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.405628    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:52:15.407025 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.406997    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8n858" event={"ID":"9262d9d7-02ad-4160-876d-b6fc06627736","Type":"ContainerStarted","Data":"05bb2fd0e6365861fa9c4a16d2c10115003931e708fd9dd1137a259b2c4f6cdf"}
Sep 09 06:52:15.424965 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.424632    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-984vs" podStartSLOduration=66.424608263 podStartE2EDuration="1m6.424608263s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:15.423042913 +0000 UTC m=+67.793302002" watchObservedRunningTime="2026-09-09 06:52:15.424608263 +0000 UTC m=+67.794867352"
Sep 09 06:52:15.782930 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.782893    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:15.783049 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.782949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:15.784498 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.784443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/6f96657d-2047-4456-94fb-5e5fa0d9e3db-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:15.788476 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.788452    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/6f96657d-2047-4456-94fb-5e5fa0d9e3db-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zlcml\" (UID: \"6f96657d-2047-4456-94fb-5e5fa0d9e3db\") " pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:15.802872 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.802764    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nqhdr\""
Sep 09 06:52:15.811135 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:15.811024    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zlcml"
Sep 09 06:52:16.108924 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:16.108852    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zlcml"]
Sep 09 06:52:16.414849 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:16.414463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dkrzk" event={"ID":"c5585130-4596-4fb1-8751-0dd1a062dc2e","Type":"ContainerStarted","Data":"a736e964994d4b00ea06ba7f64dfc8da262a622a033ddd5248f8311cc9e1e05c"}
Sep 09 06:52:16.414849 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:16.414531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-dkrzk" event={"ID":"c5585130-4596-4fb1-8751-0dd1a062dc2e","Type":"ContainerStarted","Data":"103e092fca2753e57a12bf9e66445f7f40bd9bbd25902de9b669e28945a6010b"}
Sep 09 06:52:16.473047 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:16.473009    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f96657d_2047_4456_94fb_5e5fa0d9e3db.slice/crio-50a5170eabb44c645da5a28f7684272361fa84aa42a3801b40ef551c1eee3e3b WatchSource:0}: Error finding container 50a5170eabb44c645da5a28f7684272361fa84aa42a3801b40ef551c1eee3e3b: Status 404 returned error can't find the container with id 50a5170eabb44c645da5a28f7684272361fa84aa42a3801b40ef551c1eee3e3b
Sep 09 06:52:17.420500 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:17.420466    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8n858" event={"ID":"9262d9d7-02ad-4160-876d-b6fc06627736","Type":"ContainerStarted","Data":"a8abc55cf28d2af1a8acfff5f4abe9d6ab65460f847df1b0c4342b3eb8d346f0"}
Sep 09 06:52:17.420970 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:17.420524    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8n858" event={"ID":"9262d9d7-02ad-4160-876d-b6fc06627736","Type":"ContainerStarted","Data":"507c81e3f69d03b23dc0e7bef0bcb9ae6472c72611014fa70e0de4567dff9b9e"}
Sep 09 06:52:17.420970 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:17.420611    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8n858"
Sep 09 06:52:17.422509 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:17.422478    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zlcml" event={"ID":"6f96657d-2047-4456-94fb-5e5fa0d9e3db","Type":"ContainerStarted","Data":"28cd3718dd165657b79c816ac4b2647e5caa04c93d48f5abd3d64deb0eb4c5fc"}
Sep 09 06:52:17.422640 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:17.422536    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zlcml" event={"ID":"6f96657d-2047-4456-94fb-5e5fa0d9e3db","Type":"ContainerStarted","Data":"50a5170eabb44c645da5a28f7684272361fa84aa42a3801b40ef551c1eee3e3b"}
Sep 09 06:52:17.441634 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:17.441569    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-dkrzk" podStartSLOduration=67.144374611 podStartE2EDuration="1m8.441553843s" totalImagesPullingTime="1.297179232s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:16.432997045 +0000 UTC m=+68.803256144" watchObservedRunningTime="2026-09-09 06:52:17.441553843 +0000 UTC m=+69.811812931"
Sep 09 06:52:17.441936 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:17.441898    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8n858" podStartSLOduration=33.359195339 podStartE2EDuration="35.441888794s" totalImagesPullingTime="2.082693455s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:17.440738245 +0000 UTC m=+69.810997334" watchObservedRunningTime="2026-09-09 06:52:17.441888794 +0000 UTC m=+69.812147882"
Sep 09 06:52:18.112679 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:18.112647    2563 scope.go:117] "RemoveContainer" containerID="609fa502e96049e29f4a4f3061eeab35c21b8ad41b83a1747f8fc5163532ba46"
Sep 09 06:52:18.428712 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:18.428681    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 06:52:18.429896 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:18.429143    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" event={"ID":"b1881221-61f9-4b88-8296-96fc2078afa5","Type":"ContainerStarted","Data":"044b70a107f96749e7dc69604acb7566b62c782a551d277b3389ec15b136a264"}
Sep 09 06:52:18.431857 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:18.429437    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:52:18.434490 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:18.433330    2563 patch_prober.go:28] interesting pod/console-operator-85b454874b-zgbdk container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.133.0.11:8443/readyz\": dial tcp 10.133.0.11:8443: connect: connection refused" start-of-body=
Sep 09 06:52:18.434490 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:18.433393    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" podUID="b1881221-61f9-4b88-8296-96fc2078afa5" containerName="console-operator" probeResult="failure" output="Get \"https://10.133.0.11:8443/readyz\": dial tcp 10.133.0.11:8443: connect: connection refused"
Sep 09 06:52:18.451815 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:18.451722    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-85b454874b-zgbdk" podStartSLOduration=20.730809367 podStartE2EDuration="25.45170365s" totalImagesPullingTime="4.720894283s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:18.450636345 +0000 UTC m=+70.820895434" watchObservedRunningTime="2026-09-09 06:52:18.45170365 +0000 UTC m=+70.821962739"
Sep 09 06:52:19.435177 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:19.435133    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zlcml" event={"ID":"6f96657d-2047-4456-94fb-5e5fa0d9e3db","Type":"ContainerStarted","Data":"1660eb5828d8b04f48043e8a75948c141260fd4363b7d640a2d6df6a8be6795e"}
Sep 09 06:52:19.437082 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:19.437052    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7px44" event={"ID":"857657b9-98d6-4f08-84cc-c832fa9d79ab","Type":"ContainerStarted","Data":"6f95d599dfff77b6f537a5ff42c62fe690c10eb91ad01fb335b996a6a89d1cf7"}
Sep 09 06:52:19.453768 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:19.453720    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7px44" podStartSLOduration=66.422943756 podStartE2EDuration="1m10.453703926s" totalImagesPullingTime="4.03076017s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:19.45303176 +0000 UTC m=+71.823290849" watchObservedRunningTime="2026-09-09 06:52:19.453703926 +0000 UTC m=+71.823963024"
Sep 09 06:52:19.497218 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:19.497188    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-85b454874b-zgbdk"
Sep 09 06:52:19.703802 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:19.703727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:19.804251 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:19.804214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:19.903770 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:19.903743    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:20.005399 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:20.005324    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:20.168183 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:20.168151    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:20.357049 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:20.356977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:20.417406 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:20.417377    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:20.442129 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:20.442100    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zlcml" event={"ID":"6f96657d-2047-4456-94fb-5e5fa0d9e3db","Type":"ContainerStarted","Data":"a8d209d5b4b5227d809f6ebccb667d984d694ef040a84444267aefcc4ab4d494"}
Sep 09 06:52:20.463331 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:20.463285    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zlcml" podStartSLOduration=18.201692807 podStartE2EDuration="21.463265235s" totalImagesPullingTime="3.261572428s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 06:51:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:20.462357873 +0000 UTC m=+72.832616961" watchObservedRunningTime="2026-09-09 06:52:20.463265235 +0000 UTC m=+72.833524322"
Sep 09 06:52:20.810547 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:20.810512    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:21.581861 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:21.581827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:21.798959 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:21.798922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:21.986294 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:21.986263    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:22.049159 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:22.049122    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:22.111573 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:22.111529    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:24.795015 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.794988    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"]
Sep 09 06:52:24.799328 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.799310    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"
Sep 09 06:52:24.803837 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.803816    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 09 06:52:24.805146 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.805125    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gmghs\""
Sep 09 06:52:24.805247 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.805129    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 09 06:52:24.805247 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.805182    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 09 06:52:24.805423 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.805398    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 09 06:52:24.827204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.827162    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"]
Sep 09 06:52:24.864786 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.864757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b92f\" (UniqueName: \"kubernetes.io/projected/9aaf176c-a37e-4b07-9235-166acec3eab0-kube-api-access-2b92f\") pod \"managed-serviceaccount-addon-agent-68c8cd8868-j75p4\" (UID: \"9aaf176c-a37e-4b07-9235-166acec3eab0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"
Sep 09 06:52:24.864941 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.864798    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9aaf176c-a37e-4b07-9235-166acec3eab0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68c8cd8868-j75p4\" (UID: \"9aaf176c-a37e-4b07-9235-166acec3eab0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"
Sep 09 06:52:24.895222 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.895191    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-85d98c998f-rnpv4"]
Sep 09 06:52:24.898227 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.898211    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-rnpv4"
Sep 09 06:52:24.902343 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.902321    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 09 06:52:24.902825 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.902806    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-rjmqh\""
Sep 09 06:52:24.903769 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.903748    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 09 06:52:24.947050 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.947020    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-rnpv4"]
Sep 09 06:52:24.965777 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.965751    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b92f\" (UniqueName: \"kubernetes.io/projected/9aaf176c-a37e-4b07-9235-166acec3eab0-kube-api-access-2b92f\") pod \"managed-serviceaccount-addon-agent-68c8cd8868-j75p4\" (UID: \"9aaf176c-a37e-4b07-9235-166acec3eab0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"
Sep 09 06:52:24.965934 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.965788    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgbf6\" (UniqueName: \"kubernetes.io/projected/f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2-kube-api-access-bgbf6\") pod \"downloads-85d98c998f-rnpv4\" (UID: \"f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2\") " pod="openshift-console/downloads-85d98c998f-rnpv4"
Sep 09 06:52:24.965934 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.965808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9aaf176c-a37e-4b07-9235-166acec3eab0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68c8cd8868-j75p4\" (UID: \"9aaf176c-a37e-4b07-9235-166acec3eab0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"
Sep 09 06:52:24.968538 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.968513    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9aaf176c-a37e-4b07-9235-166acec3eab0-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-68c8cd8868-j75p4\" (UID: \"9aaf176c-a37e-4b07-9235-166acec3eab0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"
Sep 09 06:52:24.972329 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.972303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:24.972713 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.972693    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8b947fcd8-zm48h"]
Sep 09 06:52:24.975996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.975981    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:24.978626 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.978608    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 09 06:52:24.978720 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.978605    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 09 06:52:24.979996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.979936    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 09 06:52:24.979996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.979985    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-4qwcc\""
Sep 09 06:52:24.987124 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.987106    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 09 06:52:24.989293 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.989270    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b92f\" (UniqueName: \"kubernetes.io/projected/9aaf176c-a37e-4b07-9235-166acec3eab0-kube-api-access-2b92f\") pod \"managed-serviceaccount-addon-agent-68c8cd8868-j75p4\" (UID: \"9aaf176c-a37e-4b07-9235-166acec3eab0\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"
Sep 09 06:52:24.997057 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:24.997040    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8b947fcd8-zm48h"]
Sep 09 06:52:25.061764 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.061673    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:25.066612 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066580    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38060476-a2c7-4787-bcf1-39187f6f88d7-installation-pull-secrets\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.066698 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066662    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-registry-tls\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.066698 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066684    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-bound-sa-token\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.066771 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066716    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-txbt7\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-kube-api-access-txbt7\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.066771 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066760    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38060476-a2c7-4787-bcf1-39187f6f88d7-registry-certificates\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.066845 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066785    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgbf6\" (UniqueName: \"kubernetes.io/projected/f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2-kube-api-access-bgbf6\") pod \"downloads-85d98c998f-rnpv4\" (UID: \"f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2\") " pod="openshift-console/downloads-85d98c998f-rnpv4"
Sep 09 06:52:25.066845 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066802    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38060476-a2c7-4787-bcf1-39187f6f88d7-ca-trust-extracted\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.066845 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38060476-a2c7-4787-bcf1-39187f6f88d7-trusted-ca\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.066943 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.066884    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38060476-a2c7-4787-bcf1-39187f6f88d7-image-registry-private-configuration\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.075341 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.075306    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgbf6\" (UniqueName: \"kubernetes.io/projected/f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2-kube-api-access-bgbf6\") pod \"downloads-85d98c998f-rnpv4\" (UID: \"f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2\") " pod="openshift-console/downloads-85d98c998f-rnpv4"
Sep 09 06:52:25.129557 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.129526    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"
Sep 09 06:52:25.160439 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.160404    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:25.167434 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.167397    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38060476-a2c7-4787-bcf1-39187f6f88d7-ca-trust-extracted\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.167599 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.167447    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38060476-a2c7-4787-bcf1-39187f6f88d7-trusted-ca\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.167599 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.167492    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38060476-a2c7-4787-bcf1-39187f6f88d7-image-registry-private-configuration\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.167599 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.167528    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38060476-a2c7-4787-bcf1-39187f6f88d7-installation-pull-secrets\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.167599 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.167562    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-registry-tls\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.167827 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.167603    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-bound-sa-token\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.167827 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.167633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-txbt7\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-kube-api-access-txbt7\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.167827 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.167692    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38060476-a2c7-4787-bcf1-39187f6f88d7-registry-certificates\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.168402 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.168211    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/38060476-a2c7-4787-bcf1-39187f6f88d7-ca-trust-extracted\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.168838 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.168790    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/38060476-a2c7-4787-bcf1-39187f6f88d7-registry-certificates\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.169029 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.169010    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/38060476-a2c7-4787-bcf1-39187f6f88d7-trusted-ca\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.170825 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.170800    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/38060476-a2c7-4787-bcf1-39187f6f88d7-installation-pull-secrets\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.170900 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.170838    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/38060476-a2c7-4787-bcf1-39187f6f88d7-image-registry-private-configuration\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.171067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.171046    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-registry-tls\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.180882 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.180859    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-bound-sa-token\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.181479 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.181227    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-txbt7\" (UniqueName: \"kubernetes.io/projected/38060476-a2c7-4787-bcf1-39187f6f88d7-kube-api-access-txbt7\") pod \"image-registry-8b947fcd8-zm48h\" (UID: \"38060476-a2c7-4787-bcf1-39187f6f88d7\") " pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.206732 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.206701    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-85d98c998f-rnpv4"
Sep 09 06:52:25.265613 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.265527    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:25.268838 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.268800    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:25.269523 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.269505    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e666318d-cdcd-483c-b61f-638bccc6cacf-service-ca-bundle\") pod \"router-default-58674b4548-wzsrc\" (UID: \"e666318d-cdcd-483c-b61f-638bccc6cacf\") " pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:25.288878 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.288857    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:25.361927 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.361889    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:25.401105 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.401082    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4"]
Sep 09 06:52:25.403707 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:25.403683    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9aaf176c_a37e_4b07_9235_166acec3eab0.slice/crio-e81944b3a3fdfa38363e9cd18e78c9f0dee61102b432f15b08a4546b66a5c92d WatchSource:0}: Error finding container e81944b3a3fdfa38363e9cd18e78c9f0dee61102b432f15b08a4546b66a5c92d: Status 404 returned error can't find the container with id e81944b3a3fdfa38363e9cd18e78c9f0dee61102b432f15b08a4546b66a5c92d
Sep 09 06:52:25.458885 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.458850    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4" event={"ID":"9aaf176c-a37e-4b07-9235-166acec3eab0","Type":"ContainerStarted","Data":"e81944b3a3fdfa38363e9cd18e78c9f0dee61102b432f15b08a4546b66a5c92d"}
Sep 09 06:52:25.478038 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.478019    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-85d98c998f-rnpv4"]
Sep 09 06:52:25.479913 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:25.479890    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5bdc63f_58d6_400e_b84f_bbdb3e1dfbc2.slice/crio-0af3c977b0c7d2145a9e23b4213be81fe674f66132fe689f0764190ed4ed378b WatchSource:0}: Error finding container 0af3c977b0c7d2145a9e23b4213be81fe674f66132fe689f0764190ed4ed378b: Status 404 returned error can't find the container with id 0af3c977b0c7d2145a9e23b4213be81fe674f66132fe689f0764190ed4ed378b
Sep 09 06:52:25.500731 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.500712    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-wmx4b\""
Sep 09 06:52:25.508755 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.508738    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:25.558916 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.558602    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8b947fcd8-zm48h"]
Sep 09 06:52:25.563686 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:25.563644    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38060476_a2c7_4787_bcf1_39187f6f88d7.slice/crio-ebbdd5ea0fda0ec2a0103ab7ed8e98e2aabb454c2c8b7de52ebee112326806f2 WatchSource:0}: Error finding container ebbdd5ea0fda0ec2a0103ab7ed8e98e2aabb454c2c8b7de52ebee112326806f2: Status 404 returned error can't find the container with id ebbdd5ea0fda0ec2a0103ab7ed8e98e2aabb454c2c8b7de52ebee112326806f2
Sep 09 06:52:25.792387 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:25.792352    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-58674b4548-wzsrc"]
Sep 09 06:52:25.796386 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:25.796360    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode666318d_cdcd_483c_b61f_638bccc6cacf.slice/crio-23d947f8ac026b1948e0d150e4b8f6929b2363fccb1ae5c4ca093b10b678647d WatchSource:0}: Error finding container 23d947f8ac026b1948e0d150e4b8f6929b2363fccb1ae5c4ca093b10b678647d: Status 404 returned error can't find the container with id 23d947f8ac026b1948e0d150e4b8f6929b2363fccb1ae5c4ca093b10b678647d
Sep 09 06:52:26.276915 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.276878    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:26.466674 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.466628    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58674b4548-wzsrc" event={"ID":"e666318d-cdcd-483c-b61f-638bccc6cacf","Type":"ContainerStarted","Data":"60d8a432d0e8f8b2fd66b912ddcb91d6392956d5551fb5713a0af51b50869577"}
Sep 09 06:52:26.466859 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.466688    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-58674b4548-wzsrc" event={"ID":"e666318d-cdcd-483c-b61f-638bccc6cacf","Type":"ContainerStarted","Data":"23d947f8ac026b1948e0d150e4b8f6929b2363fccb1ae5c4ca093b10b678647d"}
Sep 09 06:52:26.469274 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.469242    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-rnpv4" event={"ID":"f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2","Type":"ContainerStarted","Data":"0af3c977b0c7d2145a9e23b4213be81fe674f66132fe689f0764190ed4ed378b"}
Sep 09 06:52:26.471488 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.471458    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8b947fcd8-zm48h" event={"ID":"38060476-a2c7-4787-bcf1-39187f6f88d7","Type":"ContainerStarted","Data":"ef2405096a317bbea184e07d26cfa01357d29d838eaacfb08a66a12dafd2211e"}
Sep 09 06:52:26.471616 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.471504    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8b947fcd8-zm48h" event={"ID":"38060476-a2c7-4787-bcf1-39187f6f88d7","Type":"ContainerStarted","Data":"ebbdd5ea0fda0ec2a0103ab7ed8e98e2aabb454c2c8b7de52ebee112326806f2"}
Sep 09 06:52:26.472015 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.471995    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:26.489638 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.489389    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-58674b4548-wzsrc" podStartSLOduration=33.489369706 podStartE2EDuration="33.489369706s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:51:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:26.487954989 +0000 UTC m=+78.858214077" watchObservedRunningTime="2026-09-09 06:52:26.489369706 +0000 UTC m=+78.859628794"
Sep 09 06:52:26.510394 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.509417    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8b947fcd8-zm48h" podStartSLOduration=2.509387979 podStartE2EDuration="2.509387979s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:26.508345181 +0000 UTC m=+78.878604270" watchObservedRunningTime="2026-09-09 06:52:26.509387979 +0000 UTC m=+78.879647065"
Sep 09 06:52:26.510394 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.509496    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:26.512878 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.512707    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:26.682263 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.682216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:26.800775 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:26.800742    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:27.431537 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:27.431435    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8n858"
Sep 09 06:52:27.478154 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:27.477892    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:27.479459 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:27.479248    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-58674b4548-wzsrc"
Sep 09 06:52:29.206886 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:29.206846    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:29.310022 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:29.309995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:29.395018 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:29.394983    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:29.485850 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:29.485756    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4" event={"ID":"9aaf176c-a37e-4b07-9235-166acec3eab0","Type":"ContainerStarted","Data":"71256ffbe71224dc1a5f3309b7be76f87f7eab9ddb762bfab4b703448f1dd3db"}
Sep 09 06:52:29.546501 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:29.546432    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-68c8cd8868-j75p4" podStartSLOduration=2.478757688 podStartE2EDuration="5.546410339s" totalImagesPullingTime="3.067652651s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:29.543666256 +0000 UTC m=+81.913925344" watchObservedRunningTime="2026-09-09 06:52:29.546410339 +0000 UTC m=+81.916669427"
Sep 09 06:52:29.628202 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:29.628168    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:31.392393 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:31.392278    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:31.493230 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:31.493196    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:32.424552 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:32.424516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:32.496039 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:32.496002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:32.754879 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:32.754793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:34.147082 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:34.147051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:34.213745 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:34.213712    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:35.134791 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.134723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:35.226809 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.226780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:35.327522 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.327487    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:35.484663 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.484619    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:35.582890 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.582856    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7f55b8bc68-9wqmd"]
Sep 09 06:52:35.585778 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.585750    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.588404 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.588382    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 06:52:35.588506 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.588419    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 06:52:35.589785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.589298    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 06:52:35.589785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.589340    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 06:52:35.589785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.589366    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v2kwc\""
Sep 09 06:52:35.589785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.589341    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 06:52:35.593213 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.593194    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 06:52:35.597662 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.597638    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f55b8bc68-9wqmd"]
Sep 09 06:52:35.755649 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.755547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-trusted-ca-bundle\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.755649 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.755627    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9zr4\" (UniqueName: \"kubernetes.io/projected/1f779d7b-d040-438e-8e11-f464d4bb01b9-kube-api-access-s9zr4\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.755850 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.755652    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-oauth-serving-cert\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.755850 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.755679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-config\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.755850 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.755735    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-serving-cert\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.755850 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.755780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-oauth-config\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.755850 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.755822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-service-ca\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.856869 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.856836    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-trusted-ca-bundle\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.856899    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9zr4\" (UniqueName: \"kubernetes.io/projected/1f779d7b-d040-438e-8e11-f464d4bb01b9-kube-api-access-s9zr4\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.856936    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-oauth-serving-cert\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.856962    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-config\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.856984    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-serving-cert\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.857007    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-oauth-config\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.857047    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-service-ca\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857822 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.857797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-oauth-serving-cert\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857948 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.857856    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-trusted-ca-bundle\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.857948 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.857874    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-config\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.858058 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.857980    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-service-ca\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.859769 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.859751    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-serving-cert\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.859925 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.859901    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-oauth-config\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.866473 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.866448    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9zr4\" (UniqueName: \"kubernetes.io/projected/1f779d7b-d040-438e-8e11-f464d4bb01b9-kube-api-access-s9zr4\") pod \"console-7f55b8bc68-9wqmd\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") " pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:35.897292 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:35.897271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:37.604059 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.604023    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:37.756450 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.756415    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:37.810801 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.810766    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hc4s4"]
Sep 09 06:52:37.813777 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.813749    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.815970 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.815943    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 09 06:52:37.817047 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.817023    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 09 06:52:37.817047 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.817028    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 09 06:52:37.817232 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.817080    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 09 06:52:37.817931 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.817913    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 09 06:52:37.818043 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.817964    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 09 06:52:37.818043 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.817977    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-97pmt\""
Sep 09 06:52:37.864555 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.864476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:37.956306 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.956275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:37.974779 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.974744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-metrics-client-ca\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.974944 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.974790    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.974944 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.974819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-root\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.974944 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.974845    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-accelerators-collector-config\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.974944 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.974879    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-tls\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.974944 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.974903    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2fm8\" (UniqueName: \"kubernetes.io/projected/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-kube-api-access-l2fm8\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.975209 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.974973    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-sys\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.975209 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.975018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-wtmp\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:37.975209 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:37.975037    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-textfile\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.055937 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.055897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:38.076426 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076386    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-metrics-client-ca\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076567 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076436    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076567 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076469    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-root\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076567 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076496    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-accelerators-collector-config\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076567 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076526    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-tls\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076567 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2fm8\" (UniqueName: \"kubernetes.io/projected/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-kube-api-access-l2fm8\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076846 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076595    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-root\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076846 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076619    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-sys\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076846 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076655    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-wtmp\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076846 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076681    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-textfile\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.076846 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-sys\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.077100 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076897    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-wtmp\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.077100 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.076996    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-textfile\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.077202 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.077142    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-metrics-client-ca\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.077492 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.077458    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-accelerators-collector-config\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.079109 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.079088    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.079305 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.079279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-node-exporter-tls\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.086531 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.086490    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2fm8\" (UniqueName: \"kubernetes.io/projected/5c06c26c-4ba8-4f7a-90b2-48eb2e25482c-kube-api-access-l2fm8\") pod \"node-exporter-hc4s4\" (UID: \"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c\") " pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.125725 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.125645    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hc4s4"
Sep 09 06:52:38.159492 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.159461    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:38.258437 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.258402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:38.366180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.366143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:38.869261 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.869232    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:38.974066 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.974034    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:38.979763 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.979731    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 06:52:38.983494 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.983472    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:38.985858 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.985831    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 09 06:52:38.987390 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.987370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 09 06:52:38.987485 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.987370    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 09 06:52:38.987705 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.987682    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 09 06:52:38.988376 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.988353    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 09 06:52:38.988469 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.988404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 09 06:52:38.988469 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.988443    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 09 06:52:38.988581 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.988404    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 09 06:52:38.988581 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.988575    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-9b6rg\""
Sep 09 06:52:38.988724 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.988599    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 09 06:52:38.999195 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:38.999173    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 06:52:39.084640 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084604    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-web-config\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.084640 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084644    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/62f67860-882f-4c2a-a6f1-1748815a0b21-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.084880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084670    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.084880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084704    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62f67860-882f-4c2a-a6f1-1748815a0b21-config-out\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.084880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084781    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-config-volume\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.084880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.085099 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084880    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62f67860-882f-4c2a-a6f1-1748815a0b21-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.085099 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084943    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.085099 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084972    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.085099 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.084989    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.085099 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.085018    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62f67860-882f-4c2a-a6f1-1748815a0b21-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.085099 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.085066    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9kn98\" (UniqueName: \"kubernetes.io/projected/62f67860-882f-4c2a-a6f1-1748815a0b21-kube-api-access-9kn98\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.085405 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.085143    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62f67860-882f-4c2a-a6f1-1748815a0b21-tls-assets\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186545 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186512    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-web-config\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186545 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186549    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/62f67860-882f-4c2a-a6f1-1748815a0b21-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186568    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186609    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62f67860-882f-4c2a-a6f1-1748815a0b21-config-out\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186637    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-config-volume\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186670    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186713    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62f67860-882f-4c2a-a6f1-1748815a0b21-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186759    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186788    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.186818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186809    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.187220 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62f67860-882f-4c2a-a6f1-1748815a0b21-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.187220 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186872    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9kn98\" (UniqueName: \"kubernetes.io/projected/62f67860-882f-4c2a-a6f1-1748815a0b21-kube-api-access-9kn98\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.187220 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.186898    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62f67860-882f-4c2a-a6f1-1748815a0b21-tls-assets\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.187220 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.187003    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/62f67860-882f-4c2a-a6f1-1748815a0b21-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.188013 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.187960    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/62f67860-882f-4c2a-a6f1-1748815a0b21-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.188755 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:39.188606    2563 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 09 06:52:39.188755 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.188712    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/62f67860-882f-4c2a-a6f1-1748815a0b21-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.188911 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:52:39.188807    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-main-tls podName:62f67860-882f-4c2a-a6f1-1748815a0b21 nodeName:}" failed. No retries permitted until 2026-09-09 06:52:39.688786292 +0000 UTC m=+92.059045367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "62f67860-882f-4c2a-a6f1-1748815a0b21") : secret "alertmanager-main-tls" not found
Sep 09 06:52:39.190194 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.190166    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.190312 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.190194    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/62f67860-882f-4c2a-a6f1-1748815a0b21-tls-assets\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.190374 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.190336    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.191057 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.191002    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.191153 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.191133    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/62f67860-882f-4c2a-a6f1-1748815a0b21-config-out\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.191470 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.191445    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-web-config\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.192169 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.192144    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.192303 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.192282    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-config-volume\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.195458 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.195434    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9kn98\" (UniqueName: \"kubernetes.io/projected/62f67860-882f-4c2a-a6f1-1748815a0b21-kube-api-access-9kn98\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.409711 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.409667    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:39.511145 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.511048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:39.691510 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.691476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.694060 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.694037    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/62f67860-882f-4c2a-a6f1-1748815a0b21-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"62f67860-882f-4c2a-a6f1-1748815a0b21\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.895411 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.895322    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 09 06:52:39.977031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:39.976997    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:40.074468 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:40.074427    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:40.602444 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:40.602405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:40.700679 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:40.700633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:40.919992 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:40.919903    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:40.998685 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:40.998651    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:41.000243 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.000219    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"]
Sep 09 06:52:41.003606 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.003567    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.006547 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.006227    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
Sep 09 06:52:41.006547 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.006325    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
Sep 09 06:52:41.006547 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.006404    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
Sep 09 06:52:41.006547 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.006453    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
Sep 09 06:52:41.006819 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.006653    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-z6hqb\""
Sep 09 06:52:41.006819 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.006683    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
Sep 09 06:52:41.006819 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.006715    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fkl6ma3a0kj5u\""
Sep 09 06:52:41.015964 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.015942    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"]
Sep 09 06:52:41.102230 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.102187    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-metrics-client-ca\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.102396 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.102244    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.102396 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.102332    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-grpc-tls\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.102396 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.102379    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.102526 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.102446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-tls\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.102526 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.102494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.102646 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.102547    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gzlh\" (UniqueName: \"kubernetes.io/projected/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-kube-api-access-8gzlh\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.102646 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.102599    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.203230 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.203192    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-tls\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.203230 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.203238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.203476 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.203282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gzlh\" (UniqueName: \"kubernetes.io/projected/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-kube-api-access-8gzlh\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.203476 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.203310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.203476 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.203452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-metrics-client-ca\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.203615 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.203499    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.203615 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.203538    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-grpc-tls\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.203615 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.203563    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.204355 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.204297    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-metrics-client-ca\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.206518 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.206491    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-tls\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.206654 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.206563    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.206815 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.206793    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-grpc-tls\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.207032 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.207004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.207121 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.207031    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.207220 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.207201    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.211124 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.211105    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gzlh\" (UniqueName: \"kubernetes.io/projected/1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb-kube-api-access-8gzlh\") pod \"thanos-querier-545ffcc9b5-wzl6b\" (UID: \"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb\") " pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.317003 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.316971    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:41.336759 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.336727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:41.436533 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.436502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:41.537872 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:41.537842    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c06c26c_4ba8_4f7a_90b2_48eb2e25482c.slice/crio-ce811eef34a9b2c7f1411abc3eee248e69ae8b1834fb7a759624a3360563f67a WatchSource:0}: Error finding container ce811eef34a9b2c7f1411abc3eee248e69ae8b1834fb7a759624a3360563f67a: Status 404 returned error can't find the container with id ce811eef34a9b2c7f1411abc3eee248e69ae8b1834fb7a759624a3360563f67a
Sep 09 06:52:41.813204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.813007    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7f55b8bc68-9wqmd"]
Sep 09 06:52:41.816132 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:41.816105    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f779d7b_d040_438e_8e11_f464d4bb01b9.slice/crio-ae69a11dbd2a9567db239855198029ffde82343ede5632f410c6a24373f68023 WatchSource:0}: Error finding container ae69a11dbd2a9567db239855198029ffde82343ede5632f410c6a24373f68023: Status 404 returned error can't find the container with id ae69a11dbd2a9567db239855198029ffde82343ede5632f410c6a24373f68023
Sep 09 06:52:41.819836 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.819811    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"]
Sep 09 06:52:41.825736 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:41.825699    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 09 06:52:41.827257 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:41.827230    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bf3d470_23ce_4b62_9e94_4e7b7ff70ddb.slice/crio-d73be27fc87d6ea1dd1d04564b644844c28c60e3c240404089d7574935ec0b62 WatchSource:0}: Error finding container d73be27fc87d6ea1dd1d04564b644844c28c60e3c240404089d7574935ec0b62: Status 404 returned error can't find the container with id d73be27fc87d6ea1dd1d04564b644844c28c60e3c240404089d7574935ec0b62
Sep 09 06:52:41.828519 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:41.828494    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f67860_882f_4c2a_a6f1_1748815a0b21.slice/crio-090957e01e919abe180e1d9f452de8333461dcd29c89d41bab98c66d3db5e0e1 WatchSource:0}: Error finding container 090957e01e919abe180e1d9f452de8333461dcd29c89d41bab98c66d3db5e0e1: Status 404 returned error can't find the container with id 090957e01e919abe180e1d9f452de8333461dcd29c89d41bab98c66d3db5e0e1
Sep 09 06:52:42.539410 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.539367    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f55b8bc68-9wqmd" event={"ID":"1f779d7b-d040-438e-8e11-f464d4bb01b9","Type":"ContainerStarted","Data":"ae69a11dbd2a9567db239855198029ffde82343ede5632f410c6a24373f68023"}
Sep 09 06:52:42.543295 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.542822    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-85d98c998f-rnpv4" event={"ID":"f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2","Type":"ContainerStarted","Data":"3d81fccd1abbc24b3a5ce61715f625af0dede5603d7b5b3a382f58dda17f00a6"}
Sep 09 06:52:42.544512 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.543677    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-85d98c998f-rnpv4"
Sep 09 06:52:42.546790 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.546126    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hc4s4" event={"ID":"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c","Type":"ContainerStarted","Data":"ce811eef34a9b2c7f1411abc3eee248e69ae8b1834fb7a759624a3360563f67a"}
Sep 09 06:52:42.547892 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.547846    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-85d98c998f-rnpv4"
Sep 09 06:52:42.552395 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.552361    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62f67860-882f-4c2a-a6f1-1748815a0b21","Type":"ContainerStarted","Data":"090957e01e919abe180e1d9f452de8333461dcd29c89d41bab98c66d3db5e0e1"}
Sep 09 06:52:42.558801 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.558769    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b" event={"ID":"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb","Type":"ContainerStarted","Data":"d73be27fc87d6ea1dd1d04564b644844c28c60e3c240404089d7574935ec0b62"}
Sep 09 06:52:42.602206 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.601644    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-85d98c998f-rnpv4" podStartSLOduration=2.447535109 podStartE2EDuration="18.601619653s" totalImagesPullingTime="16.154084544s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:42.576700201 +0000 UTC m=+94.946959305" watchObservedRunningTime="2026-09-09 06:52:42.601619653 +0000 UTC m=+94.971878740"
Sep 09 06:52:42.605525 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.605488    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"]
Sep 09 06:52:42.610680 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.610652    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:42.613693 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.613364    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-g658s\""
Sep 09 06:52:42.615337 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.615091    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Sep 09 06:52:42.624483 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.624399    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"]
Sep 09 06:52:42.628134 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.628112    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f55b8bc68-9wqmd"]
Sep 09 06:52:42.720570 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.720530    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/78be706c-fa6f-4c1a-b04b-874bd45d97c4-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-t65fn\" (UID: \"78be706c-fa6f-4c1a-b04b-874bd45d97c4\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:42.720769 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.720686    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mgfwt\" (UniqueName: \"kubernetes.io/projected/78be706c-fa6f-4c1a-b04b-874bd45d97c4-kube-api-access-mgfwt\") pod \"monitoring-plugin-85bbbbbf4-t65fn\" (UID: \"78be706c-fa6f-4c1a-b04b-874bd45d97c4\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:42.821968 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.821268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mgfwt\" (UniqueName: \"kubernetes.io/projected/78be706c-fa6f-4c1a-b04b-874bd45d97c4-kube-api-access-mgfwt\") pod \"monitoring-plugin-85bbbbbf4-t65fn\" (UID: \"78be706c-fa6f-4c1a-b04b-874bd45d97c4\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:42.821968 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.821349    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/78be706c-fa6f-4c1a-b04b-874bd45d97c4-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-t65fn\" (UID: \"78be706c-fa6f-4c1a-b04b-874bd45d97c4\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:42.825184 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.825155    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/78be706c-fa6f-4c1a-b04b-874bd45d97c4-monitoring-plugin-cert\") pod \"monitoring-plugin-85bbbbbf4-t65fn\" (UID: \"78be706c-fa6f-4c1a-b04b-874bd45d97c4\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:42.831112 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.831027    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mgfwt\" (UniqueName: \"kubernetes.io/projected/78be706c-fa6f-4c1a-b04b-874bd45d97c4-kube-api-access-mgfwt\") pod \"monitoring-plugin-85bbbbbf4-t65fn\" (UID: \"78be706c-fa6f-4c1a-b04b-874bd45d97c4\") " pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:42.937745 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.937656    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:42.952329 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:42.952246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:43.059725 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:43.059691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:43.261876 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:43.261800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"]
Sep 09 06:52:43.265639 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:52:43.265576    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78be706c_fa6f_4c1a_b04b_874bd45d97c4.slice/crio-41e260e7af077771f20ce8339843bff01c0c60951fea1614eb70d3b5c0148a8d WatchSource:0}: Error finding container 41e260e7af077771f20ce8339843bff01c0c60951fea1614eb70d3b5c0148a8d: Status 404 returned error can't find the container with id 41e260e7af077771f20ce8339843bff01c0c60951fea1614eb70d3b5c0148a8d
Sep 09 06:52:43.569233 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:43.569194    2563 generic.go:356] "Generic (PLEG): container finished" podID="5c06c26c-4ba8-4f7a-90b2-48eb2e25482c" containerID="2d745b4114f21ebacfbadfd54dc81613afe9c667c2bc6c3a9a0e6bb2fdef92d1" exitCode=0
Sep 09 06:52:43.569776 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:43.569310    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hc4s4" event={"ID":"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c","Type":"ContainerDied","Data":"2d745b4114f21ebacfbadfd54dc81613afe9c667c2bc6c3a9a0e6bb2fdef92d1"}
Sep 09 06:52:43.579375 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:43.579278    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn" event={"ID":"78be706c-fa6f-4c1a-b04b-874bd45d97c4","Type":"ContainerStarted","Data":"41e260e7af077771f20ce8339843bff01c0c60951fea1614eb70d3b5c0148a8d"}
Sep 09 06:52:44.072039 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:44.071999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:44.161575 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:44.161541    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:44.267091 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:44.267009    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:44.588252 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:44.588211    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hc4s4" event={"ID":"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c","Type":"ContainerStarted","Data":"259e12ac4abd78c0710f852358626e39158ee022c9c79270abaa0d7f40e5f783"}
Sep 09 06:52:46.418286 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:46.418249    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-984vs"
Sep 09 06:52:46.856058 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:46.856017    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:46.925740 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:46.925623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:47.607269 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:47.607186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hc4s4" event={"ID":"5c06c26c-4ba8-4f7a-90b2-48eb2e25482c","Type":"ContainerStarted","Data":"ab39e75d056b74d48b0befe7f24854c0d835828f8ef9e1aa41e216e5a1274cdb"}
Sep 09 06:52:47.628685 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:47.628622    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hc4s4" podStartSLOduration=9.208245861 podStartE2EDuration="10.628600067s" totalImagesPullingTime="1.311778498s" totalInitContainerRuntime="108.575708ms" isStatefulPod=true podCreationTimestamp="2026-09-09 06:52:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:47.626853247 +0000 UTC m=+99.997112384" watchObservedRunningTime="2026-09-09 06:52:47.628600067 +0000 UTC m=+99.998859152"
Sep 09 06:52:48.484861 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.484830    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8b947fcd8-zm48h"
Sep 09 06:52:48.612840 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.612804    2563 generic.go:356] "Generic (PLEG): container finished" podID="62f67860-882f-4c2a-a6f1-1748815a0b21" containerID="b30c8f6d54be34706afa7c1108e5983a1f3c8c6c10ccd2b03e693c455a96bfd4" exitCode=0
Sep 09 06:52:48.613271 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.612895    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62f67860-882f-4c2a-a6f1-1748815a0b21","Type":"ContainerDied","Data":"b30c8f6d54be34706afa7c1108e5983a1f3c8c6c10ccd2b03e693c455a96bfd4"}
Sep 09 06:52:48.615761 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.615737    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b" event={"ID":"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb","Type":"ContainerStarted","Data":"ad843ac6c3f542e18ff1fcdad7aa53e6c1d146bce70467d77db7012fd5d1c901"}
Sep 09 06:52:48.615879 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.615799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b" event={"ID":"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb","Type":"ContainerStarted","Data":"6d0a540e31d9a091fa3436b14528f57d7bcb45d49a46ff332f75346d38a02836"}
Sep 09 06:52:48.615879 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.615838    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b" event={"ID":"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb","Type":"ContainerStarted","Data":"d9b218f8a246d49246ef76e846969b3bfc66157bf9b3bec90e36e31429b4260e"}
Sep 09 06:52:48.618409 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.618378    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn" event={"ID":"78be706c-fa6f-4c1a-b04b-874bd45d97c4","Type":"ContainerStarted","Data":"ae279978413f5817230acf77f1511113a18fa8c23a02342dae14ea5402392cf9"}
Sep 09 06:52:48.618906 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.618855    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:48.621765 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.620784    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f55b8bc68-9wqmd" event={"ID":"1f779d7b-d040-438e-8e11-f464d4bb01b9","Type":"ContainerStarted","Data":"107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1"}
Sep 09 06:52:48.625185 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.625162    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn"
Sep 09 06:52:48.681179 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.681124    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7f55b8bc68-9wqmd" podStartSLOduration=8.244554301 podStartE2EDuration="13.681103398s" totalImagesPullingTime="5.436549097s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:48.679303511 +0000 UTC m=+101.049562598" watchObservedRunningTime="2026-09-09 06:52:48.681103398 +0000 UTC m=+101.051362486"
Sep 09 06:52:48.697149 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.697097    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-85bbbbbf4-t65fn" podStartSLOduration=1.8114412949999998 podStartE2EDuration="6.697080799s" totalImagesPullingTime="4.885639504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:48.696120629 +0000 UTC m=+101.066379719" watchObservedRunningTime="2026-09-09 06:52:48.697080799 +0000 UTC m=+101.067339889"
Sep 09 06:52:48.957696 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:48.957610    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:49.027675 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:49.027641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:50.633973 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:50.633919    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b" event={"ID":"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb","Type":"ContainerStarted","Data":"605e4dc5324575cb4bf69483bc6bc71fdedf93c5f89deb3f469b9d294fba8ed5"}
Sep 09 06:52:50.634446 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:50.634001    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b" event={"ID":"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb","Type":"ContainerStarted","Data":"3f2f67424dbc167615b2a8449444e9f45ed0807bbb33c5e0c611d8ec9dd21538"}
Sep 09 06:52:50.634446 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:50.634043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b" event={"ID":"1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb","Type":"ContainerStarted","Data":"89b935acda98c1acdeeeb0d72b24960cf2aca4eea432945a128116a2614758c6"}
Sep 09 06:52:50.659780 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:50.659684    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b" podStartSLOduration=3.859480129 podStartE2EDuration="10.659660038s" totalImagesPullingTime="6.800179909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:50.656120526 +0000 UTC m=+103.026379615" watchObservedRunningTime="2026-09-09 06:52:50.659660038 +0000 UTC m=+103.029919140"
Sep 09 06:52:51.640357 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:51.640323    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62f67860-882f-4c2a-a6f1-1748815a0b21","Type":"ContainerStarted","Data":"b2780616e31d41698ff9b4e4f0544f10f3f852e04b0e683562166159940ce7e0"}
Sep 09 06:52:51.640660 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:51.640625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:52.188317 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:52.188242    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:52.292910 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:52.292877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:52.647743 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:52.647658    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62f67860-882f-4c2a-a6f1-1748815a0b21","Type":"ContainerStarted","Data":"85c84d8ceeafdb6ac579c35d2158e311625dd2c6e1ac7dffa44001a80ba1b93b"}
Sep 09 06:52:52.647743 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:52.647731    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62f67860-882f-4c2a-a6f1-1748815a0b21","Type":"ContainerStarted","Data":"762068d7116b44ec194050a1c89d3f399c71b82f75da37a5da13658b3f674819"}
Sep 09 06:52:52.648266 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:52.647758    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62f67860-882f-4c2a-a6f1-1748815a0b21","Type":"ContainerStarted","Data":"4e5ec48f647954067bcfe8123e2b9b22bfaa76fe6035a683d2a50c13692a9ec7"}
Sep 09 06:52:52.648266 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:52.647782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62f67860-882f-4c2a-a6f1-1748815a0b21","Type":"ContainerStarted","Data":"3300a1926af14e5fe6aa747f376730860f66ea9dba7ff4ee7c051e86d40f0176"}
Sep 09 06:52:52.648266 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:52.647808    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"62f67860-882f-4c2a-a6f1-1748815a0b21","Type":"ContainerStarted","Data":"771cbbce2afb2fa8856f491bcd7e7b843f72a7308dd3fb5ae590a2a9f60b09e9"}
Sep 09 06:52:52.675035 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:52.674982    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=6.445547886 podStartE2EDuration="14.67496339s" totalImagesPullingTime="8.218829652s" totalInitContainerRuntime="10.585852ms" isStatefulPod=false podCreationTimestamp="2026-09-09 06:52:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:52:52.672548834 +0000 UTC m=+105.042807966" watchObservedRunningTime="2026-09-09 06:52:52.67496339 +0000 UTC m=+105.045222478"
Sep 09 06:52:53.396489 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:53.396457    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:53.497013 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:53.496976    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:53.597508 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:53.597478    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:55.897899 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:55.897863    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:52:57.118888 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:57.118851    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:57.178519 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:57.178488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:57.655427 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:57.655396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-545ffcc9b5-wzl6b"
Sep 09 06:52:59.075709 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:59.075674    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:52:59.142489 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:52:59.142461    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:08.557689 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:08.557655    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:13.644769 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.644721    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7f55b8bc68-9wqmd" podUID="1f779d7b-d040-438e-8e11-f464d4bb01b9" containerName="console" containerID="cri-o://107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1" gracePeriod=15
Sep 09 06:53:13.881223 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.881196    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:53:13.994528 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.994496    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-serving-cert\") pod \"1f779d7b-d040-438e-8e11-f464d4bb01b9\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") "
Sep 09 06:53:13.994738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.994540    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-config\") pod \"1f779d7b-d040-438e-8e11-f464d4bb01b9\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") "
Sep 09 06:53:13.994738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.994614    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-oauth-config\") pod \"1f779d7b-d040-438e-8e11-f464d4bb01b9\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") "
Sep 09 06:53:13.994738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.994664    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-oauth-serving-cert\") pod \"1f779d7b-d040-438e-8e11-f464d4bb01b9\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") "
Sep 09 06:53:13.994738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.994690    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-service-ca\") pod \"1f779d7b-d040-438e-8e11-f464d4bb01b9\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") "
Sep 09 06:53:13.994738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.994723    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-trusted-ca-bundle\") pod \"1f779d7b-d040-438e-8e11-f464d4bb01b9\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") "
Sep 09 06:53:13.994983 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.994796    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9zr4\" (UniqueName: \"kubernetes.io/projected/1f779d7b-d040-438e-8e11-f464d4bb01b9-kube-api-access-s9zr4\") pod \"1f779d7b-d040-438e-8e11-f464d4bb01b9\" (UID: \"1f779d7b-d040-438e-8e11-f464d4bb01b9\") "
Sep 09 06:53:13.994983 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.994924    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-config" (OuterVolumeSpecName: "console-config") pod "1f779d7b-d040-438e-8e11-f464d4bb01b9" (UID: "1f779d7b-d040-438e-8e11-f464d4bb01b9"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:53:13.995084 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.995042    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1f779d7b-d040-438e-8e11-f464d4bb01b9" (UID: "1f779d7b-d040-438e-8e11-f464d4bb01b9"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:53:13.995145 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.995066    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-service-ca" (OuterVolumeSpecName: "service-ca") pod "1f779d7b-d040-438e-8e11-f464d4bb01b9" (UID: "1f779d7b-d040-438e-8e11-f464d4bb01b9"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:53:13.995145 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.995089    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-config\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:53:13.995145 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.995131    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1f779d7b-d040-438e-8e11-f464d4bb01b9" (UID: "1f779d7b-d040-438e-8e11-f464d4bb01b9"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:53:13.996866 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.996836    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f779d7b-d040-438e-8e11-f464d4bb01b9-kube-api-access-s9zr4" (OuterVolumeSpecName: "kube-api-access-s9zr4") pod "1f779d7b-d040-438e-8e11-f464d4bb01b9" (UID: "1f779d7b-d040-438e-8e11-f464d4bb01b9"). InnerVolumeSpecName "kube-api-access-s9zr4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 06:53:13.996974 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.996879    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1f779d7b-d040-438e-8e11-f464d4bb01b9" (UID: "1f779d7b-d040-438e-8e11-f464d4bb01b9"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:53:13.996974 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:13.996894    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1f779d7b-d040-438e-8e11-f464d4bb01b9" (UID: "1f779d7b-d040-438e-8e11-f464d4bb01b9"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:53:14.074740 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.074711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:14.095918 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.095894    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9zr4\" (UniqueName: \"kubernetes.io/projected/1f779d7b-d040-438e-8e11-f464d4bb01b9-kube-api-access-s9zr4\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:53:14.095918 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.095916    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-serving-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:53:14.096031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.095925    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1f779d7b-d040-438e-8e11-f464d4bb01b9-console-oauth-config\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:53:14.096031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.095934    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-oauth-serving-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:53:14.096031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.095942    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:53:14.096031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.095949    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1f779d7b-d040-438e-8e11-f464d4bb01b9-trusted-ca-bundle\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:53:14.130929 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.130875    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:14.736082 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.736045    2563 generic.go:356] "Generic (PLEG): container finished" podID="1f779d7b-d040-438e-8e11-f464d4bb01b9" containerID="107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1" exitCode=0
Sep 09 06:53:14.736473 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.736101    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7f55b8bc68-9wqmd"
Sep 09 06:53:14.736473 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.736125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f55b8bc68-9wqmd" event={"ID":"1f779d7b-d040-438e-8e11-f464d4bb01b9","Type":"ContainerDied","Data":"107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1"}
Sep 09 06:53:14.736473 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.736190    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7f55b8bc68-9wqmd" event={"ID":"1f779d7b-d040-438e-8e11-f464d4bb01b9","Type":"ContainerDied","Data":"ae69a11dbd2a9567db239855198029ffde82343ede5632f410c6a24373f68023"}
Sep 09 06:53:14.736473 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.736211    2563 scope.go:117] "RemoveContainer" containerID="107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1"
Sep 09 06:53:14.744330 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.744309    2563 scope.go:117] "RemoveContainer" containerID="107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1"
Sep 09 06:53:14.744626 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:53:14.744601    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1\": container with ID starting with 107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1 not found: ID does not exist" containerID="107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1"
Sep 09 06:53:14.744698 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.744637    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1"} err="failed to get container status \"107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1\": rpc error: code = NotFound desc = could not find container \"107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1\": container with ID starting with 107e5c78075174415a326888804a20a4c838e9ddda35fb6f0bb841eb791579f1 not found: ID does not exist"
Sep 09 06:53:14.754252 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.754228    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7f55b8bc68-9wqmd"]
Sep 09 06:53:14.760457 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:14.760437    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7f55b8bc68-9wqmd"]
Sep 09 06:53:16.115868 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:16.115837    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f779d7b-d040-438e-8e11-f464d4bb01b9" path="/var/lib/kubelet/pods/1f779d7b-d040-438e-8e11-f464d4bb01b9/volumes"
Sep 09 06:53:21.974963 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:21.974925    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:22.075372 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:22.075336    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:22.148344 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:22.148312    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:24.780137 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:24.780103    2563 generic.go:356] "Generic (PLEG): container finished" podID="2a5575d9-f5a0-4de3-9833-f30eed7135cb" containerID="8116201e557d55f4e18d57ff938b9dad99a612d5203975d9b11fe4e3e14623b3" exitCode=0
Sep 09 06:53:24.780608 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:24.780180    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh" event={"ID":"2a5575d9-f5a0-4de3-9833-f30eed7135cb","Type":"ContainerDied","Data":"8116201e557d55f4e18d57ff938b9dad99a612d5203975d9b11fe4e3e14623b3"}
Sep 09 06:53:24.780608 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:24.780580    2563 scope.go:117] "RemoveContainer" containerID="8116201e557d55f4e18d57ff938b9dad99a612d5203975d9b11fe4e3e14623b3"
Sep 09 06:53:25.786280 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:25.786245    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-8d7dd4675-pfbvh" event={"ID":"2a5575d9-f5a0-4de3-9833-f30eed7135cb","Type":"ContainerStarted","Data":"e93476f0be54cca40b702b0ab11fc479f545eac2ac138c9ebce64cee538d534a"}
Sep 09 06:53:29.075146 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:29.075105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:29.132694 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:29.132656    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:57.223555 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:57.223519    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:53:57.515264 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:53:57.515182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:02.324600 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.324511    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f9446bc69-cddks"]
Sep 09 06:54:02.324938 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.324832    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f779d7b-d040-438e-8e11-f464d4bb01b9" containerName="console"
Sep 09 06:54:02.324938 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.324844    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f779d7b-d040-438e-8e11-f464d4bb01b9" containerName="console"
Sep 09 06:54:02.324938 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.324900    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f779d7b-d040-438e-8e11-f464d4bb01b9" containerName="console"
Sep 09 06:54:02.326842 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.326813    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.329332 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.329311    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 09 06:54:02.329440 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.329348    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 09 06:54:02.329440 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.329362    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-7hptz\""
Sep 09 06:54:02.329440 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.329417    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 09 06:54:02.329805 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.329775    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 09 06:54:02.330216 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.330196    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 09 06:54:02.333738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.333711    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 09 06:54:02.340562 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.340540    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f9446bc69-cddks"]
Sep 09 06:54:02.377825 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.377789    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-telemeter-client-tls\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.377825 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.377824    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-metrics-client-ca\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.378021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.377862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.378021 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.377971    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-serving-certs-ca-bundle\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.378088 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.378027    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-federate-client-tls\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.378088 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.378061    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26hf\" (UniqueName: \"kubernetes.io/projected/2611e206-c6c9-4302-aa21-3ae4a70155d4-kube-api-access-r26hf\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.378154 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.378083    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-secret-telemeter-client\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.378154 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.378111    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.478645 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.478572    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-telemeter-client-tls\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.478820 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.478660    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-metrics-client-ca\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.478820 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.478781    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.478905 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.478838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-serving-certs-ca-bundle\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.478905 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.478882    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-federate-client-tls\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.478996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.478909    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r26hf\" (UniqueName: \"kubernetes.io/projected/2611e206-c6c9-4302-aa21-3ae4a70155d4-kube-api-access-r26hf\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.478996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.478949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-secret-telemeter-client\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.478996 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.478976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.479437 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.479412    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-metrics-client-ca\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.479657 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.479634    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-serving-certs-ca-bundle\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.479879 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.479860    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2611e206-c6c9-4302-aa21-3ae4a70155d4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.481376 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.481357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-telemeter-client-tls\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.481679 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.481659    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-secret-telemeter-client\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.481901 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.481879    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.481962 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.481928    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/2611e206-c6c9-4302-aa21-3ae4a70155d4-federate-client-tls\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.487284 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.487263    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26hf\" (UniqueName: \"kubernetes.io/projected/2611e206-c6c9-4302-aa21-3ae4a70155d4-kube-api-access-r26hf\") pod \"telemeter-client-7f9446bc69-cddks\" (UID: \"2611e206-c6c9-4302-aa21-3ae4a70155d4\") " pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.636497 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.636414    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks"
Sep 09 06:54:02.916124 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.916073    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f9446bc69-cddks"]
Sep 09 06:54:02.918393 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:54:02.918368    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2611e206_c6c9_4302_aa21_3ae4a70155d4.slice/crio-925a153baca989e22bceb43934b9e1b613d957c7b6fc1e230194846397d9974e WatchSource:0}: Error finding container 925a153baca989e22bceb43934b9e1b613d957c7b6fc1e230194846397d9974e: Status 404 returned error can't find the container with id 925a153baca989e22bceb43934b9e1b613d957c7b6fc1e230194846397d9974e
Sep 09 06:54:02.931912 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:02.931886    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks" event={"ID":"2611e206-c6c9-4302-aa21-3ae4a70155d4","Type":"ContainerStarted","Data":"925a153baca989e22bceb43934b9e1b613d957c7b6fc1e230194846397d9974e"}
Sep 09 06:54:04.942791 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:04.942760    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks" event={"ID":"2611e206-c6c9-4302-aa21-3ae4a70155d4","Type":"ContainerStarted","Data":"8b62796d9a3c0f6bd537e03c3b44d2d8a24453751dbdaae005d08d734ac05485"}
Sep 09 06:54:04.943159 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:04.942839    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks" event={"ID":"2611e206-c6c9-4302-aa21-3ae4a70155d4","Type":"ContainerStarted","Data":"9e27f2de50880c3ee48d682bc9aa1e508c20bff804d0c8bfb69986613af943f3"}
Sep 09 06:54:04.943159 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:04.942859    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks" event={"ID":"2611e206-c6c9-4302-aa21-3ae4a70155d4","Type":"ContainerStarted","Data":"649f7599e388ef30648f557dd1a79452adcc23c7c0afde26a0df5b5994227086"}
Sep 09 06:54:04.965612 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:04.965548    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f9446bc69-cddks" podStartSLOduration=1.235669962 podStartE2EDuration="2.965532431s" totalImagesPullingTime="1.729862469s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:54:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:54:04.963926647 +0000 UTC m=+177.334185771" watchObservedRunningTime="2026-09-09 06:54:04.965532431 +0000 UTC m=+177.335791541"
Sep 09 06:54:05.047142 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.047111    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-79c79c8f59-hmqvz"]
Sep 09 06:54:05.049385 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.049355    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.051773 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.051740    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 09 06:54:05.052192 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.051895    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 09 06:54:05.052192 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.052112    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v2kwc\""
Sep 09 06:54:05.052383 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.052362    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 09 06:54:05.052662 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.052635    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 09 06:54:05.052786 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.052764    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 09 06:54:05.058230 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.058209    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 09 06:54:05.062470 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.062448    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79c79c8f59-hmqvz"]
Sep 09 06:54:05.103054 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.102839    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m48sh\" (UniqueName: \"kubernetes.io/projected/fc4f774f-24d0-445a-9e02-ee5be9b97cec-kube-api-access-m48sh\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.103054 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.102889    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-oauth-serving-cert\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.103054 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.102920    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-service-ca\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.103054 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.102963    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-oauth-config\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.103054 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.103019    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-serving-cert\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.103373 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.103106    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-config\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.103373 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.103133    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-trusted-ca-bundle\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.103855 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.103836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:05.165918 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.165894    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:05.204247 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.204182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-serving-cert\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.204247 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.204231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-config\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.204431 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.204251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-trusted-ca-bundle\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.204431 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.204297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m48sh\" (UniqueName: \"kubernetes.io/projected/fc4f774f-24d0-445a-9e02-ee5be9b97cec-kube-api-access-m48sh\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.204431 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.204327    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-oauth-serving-cert\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.204431 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.204348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-service-ca\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.204431 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.204385    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-oauth-config\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.205354 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.205151    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-config\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.205354 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.205308    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-oauth-serving-cert\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.205505 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.205353    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-service-ca\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.205505 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.205427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-trusted-ca-bundle\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.207100 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.207077    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-oauth-config\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.207299 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.207279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-serving-cert\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.214910 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.214891    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m48sh\" (UniqueName: \"kubernetes.io/projected/fc4f774f-24d0-445a-9e02-ee5be9b97cec-kube-api-access-m48sh\") pod \"console-79c79c8f59-hmqvz\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") " pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.246146 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.246116    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:05.361082 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.361036    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:05.632211 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.632141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-79c79c8f59-hmqvz"]
Sep 09 06:54:05.634943 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:54:05.634919    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc4f774f_24d0_445a_9e02_ee5be9b97cec.slice/crio-fdf60daefea4e98a51307584e15f0cdc470ace981e6c566c571e84d4ae4f2f9b WatchSource:0}: Error finding container fdf60daefea4e98a51307584e15f0cdc470ace981e6c566c571e84d4ae4f2f9b: Status 404 returned error can't find the container with id fdf60daefea4e98a51307584e15f0cdc470ace981e6c566c571e84d4ae4f2f9b
Sep 09 06:54:05.916968 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.916881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:05.948084 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.948055    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c79c8f59-hmqvz" event={"ID":"fc4f774f-24d0-445a-9e02-ee5be9b97cec","Type":"ContainerStarted","Data":"c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00"}
Sep 09 06:54:05.948465 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.948104    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c79c8f59-hmqvz" event={"ID":"fc4f774f-24d0-445a-9e02-ee5be9b97cec","Type":"ContainerStarted","Data":"fdf60daefea4e98a51307584e15f0cdc470ace981e6c566c571e84d4ae4f2f9b"}
Sep 09 06:54:05.965832 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:05.965785    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-79c79c8f59-hmqvz" podStartSLOduration=0.965768637 podStartE2EDuration="965.768637ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:54:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:54:05.964744724 +0000 UTC m=+178.335003812" watchObservedRunningTime="2026-09-09 06:54:05.965768637 +0000 UTC m=+178.336027724"
Sep 09 06:54:06.016182 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:06.016149    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:12.478663 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:12.478628    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:12.537978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:12.537948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:14.519010 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:14.518977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:15.361751 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:15.361694    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:15.361929 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:15.361807    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:15.366747 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:15.366722    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:15.994495 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:15.994465    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:16.155695 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.155661    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d444f74cd-qt9wr"]
Sep 09 06:54:16.158514 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.158488    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.161388 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.161349    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:16.168880 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.168855    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d444f74cd-qt9wr"]
Sep 09 06:54:16.297807 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.297702    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-oauth-config\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.297807 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.297749    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-console-config\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.297807 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.297780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdhbf\" (UniqueName: \"kubernetes.io/projected/be3c5945-8f13-4087-aeaf-58594305374c-kube-api-access-mdhbf\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.298035 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.297812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-oauth-serving-cert\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.298035 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.297872    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-trusted-ca-bundle\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.298035 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.297914    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-service-ca\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.298129 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.298024    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-serving-cert\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.399142 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.399096    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-oauth-config\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.399142 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.399144    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-console-config\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.399353 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.399162    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdhbf\" (UniqueName: \"kubernetes.io/projected/be3c5945-8f13-4087-aeaf-58594305374c-kube-api-access-mdhbf\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.399353 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.399182    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-oauth-serving-cert\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.399353 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.399221    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-trusted-ca-bundle\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.399353 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.399250    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-service-ca\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.399353 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.399291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-serving-cert\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.400032 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.400005    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-console-config\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.400124 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.400014    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-oauth-serving-cert\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.400124 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.400099    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-trusted-ca-bundle\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.400193 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.400120    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-service-ca\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.401638 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.401610    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-oauth-config\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.401831 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.401811    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-serving-cert\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.407868 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.407848    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdhbf\" (UniqueName: \"kubernetes.io/projected/be3c5945-8f13-4087-aeaf-58594305374c-kube-api-access-mdhbf\") pod \"console-d444f74cd-qt9wr\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") " pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.469395 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.469359    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:16.740465 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.740372    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d444f74cd-qt9wr"]
Sep 09 06:54:16.743079 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:54:16.743051    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3c5945_8f13_4087_aeaf_58594305374c.slice/crio-25e29b5cb952b6dac7344f7170996cbc5468a65b53ba2c34f312e006917f31d9 WatchSource:0}: Error finding container 25e29b5cb952b6dac7344f7170996cbc5468a65b53ba2c34f312e006917f31d9: Status 404 returned error can't find the container with id 25e29b5cb952b6dac7344f7170996cbc5468a65b53ba2c34f312e006917f31d9
Sep 09 06:54:16.990845 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.990755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:16.995479 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.995451    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d444f74cd-qt9wr" event={"ID":"be3c5945-8f13-4087-aeaf-58594305374c","Type":"ContainerStarted","Data":"b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24"}
Sep 09 06:54:16.995984 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:16.995505    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d444f74cd-qt9wr" event={"ID":"be3c5945-8f13-4087-aeaf-58594305374c","Type":"ContainerStarted","Data":"25e29b5cb952b6dac7344f7170996cbc5468a65b53ba2c34f312e006917f31d9"}
Sep 09 06:54:17.015830 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:17.015784    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d444f74cd-qt9wr" podStartSLOduration=1.015769076 podStartE2EDuration="1.015769076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:54:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:54:17.015273557 +0000 UTC m=+189.385532646" watchObservedRunningTime="2026-09-09 06:54:17.015769076 +0000 UTC m=+189.386028163"
Sep 09 06:54:17.094092 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:17.094058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:23.362457 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:23.362412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:23.428123 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:23.428091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:25.071415 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:25.071381    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:25.133141 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:25.133113    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:26.470383 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:26.470338    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:26.470786 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:26.470410    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:26.475445 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:26.475419    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:27.040160 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:27.040134    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:54:27.088896 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:27.088860    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79c79c8f59-hmqvz"]
Sep 09 06:54:27.190841 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:27.190768    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:27.301963 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:27.301877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:27.356603 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:27.356557    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:29.086842 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:29.086812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:29.202254 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:29.202223    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:54:52.112654 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.112574    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-79c79c8f59-hmqvz" podUID="fc4f774f-24d0-445a-9e02-ee5be9b97cec" containerName="console" containerID="cri-o://c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00" gracePeriod=15
Sep 09 06:54:52.363061 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.363003    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:52.390449 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.390417    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-serving-cert\") pod \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") "
Sep 09 06:54:52.390601 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.390475    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-service-ca\") pod \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") "
Sep 09 06:54:52.390601 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.390506    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-oauth-config\") pod \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") "
Sep 09 06:54:52.390601 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.390529    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-config\") pod \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") "
Sep 09 06:54:52.390601 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.390557    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m48sh\" (UniqueName: \"kubernetes.io/projected/fc4f774f-24d0-445a-9e02-ee5be9b97cec-kube-api-access-m48sh\") pod \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") "
Sep 09 06:54:52.390866 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.390627    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-oauth-serving-cert\") pod \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") "
Sep 09 06:54:52.390866 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.390657    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-trusted-ca-bundle\") pod \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\" (UID: \"fc4f774f-24d0-445a-9e02-ee5be9b97cec\") "
Sep 09 06:54:52.391063 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.391021    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-service-ca" (OuterVolumeSpecName: "service-ca") pod "fc4f774f-24d0-445a-9e02-ee5be9b97cec" (UID: "fc4f774f-24d0-445a-9e02-ee5be9b97cec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:54:52.391323 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.391164    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fc4f774f-24d0-445a-9e02-ee5be9b97cec" (UID: "fc4f774f-24d0-445a-9e02-ee5be9b97cec"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:54:52.391323 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.391259    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-config" (OuterVolumeSpecName: "console-config") pod "fc4f774f-24d0-445a-9e02-ee5be9b97cec" (UID: "fc4f774f-24d0-445a-9e02-ee5be9b97cec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:54:52.391494 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.391375    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fc4f774f-24d0-445a-9e02-ee5be9b97cec" (UID: "fc4f774f-24d0-445a-9e02-ee5be9b97cec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:54:52.392690 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.392660    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fc4f774f-24d0-445a-9e02-ee5be9b97cec" (UID: "fc4f774f-24d0-445a-9e02-ee5be9b97cec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:54:52.392826 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.392805    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fc4f774f-24d0-445a-9e02-ee5be9b97cec" (UID: "fc4f774f-24d0-445a-9e02-ee5be9b97cec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:54:52.392905 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.392885    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc4f774f-24d0-445a-9e02-ee5be9b97cec-kube-api-access-m48sh" (OuterVolumeSpecName: "kube-api-access-m48sh") pod "fc4f774f-24d0-445a-9e02-ee5be9b97cec" (UID: "fc4f774f-24d0-445a-9e02-ee5be9b97cec"). InnerVolumeSpecName "kube-api-access-m48sh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 06:54:52.491703 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.491668    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-oauth-serving-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:54:52.491703 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.491696    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-trusted-ca-bundle\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:54:52.491703 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.491706    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-serving-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:54:52.491927 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.491717    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:54:52.491927 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.491725    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-oauth-config\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:54:52.491927 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.491735    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fc4f774f-24d0-445a-9e02-ee5be9b97cec-console-config\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:54:52.491927 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:52.491743    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m48sh\" (UniqueName: \"kubernetes.io/projected/fc4f774f-24d0-445a-9e02-ee5be9b97cec-kube-api-access-m48sh\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:54:53.137793 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.137757    2563 generic.go:356] "Generic (PLEG): container finished" podID="fc4f774f-24d0-445a-9e02-ee5be9b97cec" containerID="c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00" exitCode=0
Sep 09 06:54:53.137793 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.137801    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-79c79c8f59-hmqvz"
Sep 09 06:54:53.138273 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.137815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c79c8f59-hmqvz" event={"ID":"fc4f774f-24d0-445a-9e02-ee5be9b97cec","Type":"ContainerDied","Data":"c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00"}
Sep 09 06:54:53.138273 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.137861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-79c79c8f59-hmqvz" event={"ID":"fc4f774f-24d0-445a-9e02-ee5be9b97cec","Type":"ContainerDied","Data":"fdf60daefea4e98a51307584e15f0cdc470ace981e6c566c571e84d4ae4f2f9b"}
Sep 09 06:54:53.138273 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.137881    2563 scope.go:117] "RemoveContainer" containerID="c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00"
Sep 09 06:54:53.146652 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.146637    2563 scope.go:117] "RemoveContainer" containerID="c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00"
Sep 09 06:54:53.146918 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:54:53.146900    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00\": container with ID starting with c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00 not found: ID does not exist" containerID="c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00"
Sep 09 06:54:53.146979 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.146929    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00"} err="failed to get container status \"c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00\": rpc error: code = NotFound desc = could not find container \"c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00\": container with ID starting with c2cec7ef97814f3a6e219ec1f5e61ba2bfe1383512d12d2e0b5933d4c4f38b00 not found: ID does not exist"
Sep 09 06:54:53.158071 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.158045    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-79c79c8f59-hmqvz"]
Sep 09 06:54:53.162444 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:53.162420    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-79c79c8f59-hmqvz"]
Sep 09 06:54:54.115881 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:54.115846    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc4f774f-24d0-445a-9e02-ee5be9b97cec" path="/var/lib/kubelet/pods/fc4f774f-24d0-445a-9e02-ee5be9b97cec/volumes"
Sep 09 06:54:54.946505 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:54:54.946470    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:31.244979 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:31.244906    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:31.343998 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:31.343963    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:53.244225 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:53.244190    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:54.587641 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:54.587582    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:57.048323 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:57.048281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:57.149025 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:57.148988    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:58.559845 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:58.559812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:58.660272 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:58.660239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:58.902632 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:58.902539    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:55:59.027415 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:55:59.027377    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:00.746831 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:00.746798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:00.934960 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:00.934929    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:01.036793 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.036711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:01.338583 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.338507    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"]
Sep 09 06:56:01.338866 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.338851    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc4f774f-24d0-445a-9e02-ee5be9b97cec" containerName="console"
Sep 09 06:56:01.338909 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.338869    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc4f774f-24d0-445a-9e02-ee5be9b97cec" containerName="console"
Sep 09 06:56:01.338949 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.338925    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc4f774f-24d0-445a-9e02-ee5be9b97cec" containerName="console"
Sep 09 06:56:01.341642 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.341626    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"
Sep 09 06:56:01.344362 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.344341    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 09 06:56:01.345123 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.345105    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 09 06:56:01.345123 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.345112    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-bxsbj\""
Sep 09 06:56:01.352067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.352045    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"]
Sep 09 06:56:01.456238 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.456200    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqkzl\" (UniqueName: \"kubernetes.io/projected/00966140-d092-4cd8-8794-f5e4de6a542e-kube-api-access-qqkzl\") pod \"cert-manager-cainjector-559cdd8c87-jhpwq\" (UID: \"00966140-d092-4cd8-8794-f5e4de6a542e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"
Sep 09 06:56:01.456238 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.456241    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00966140-d092-4cd8-8794-f5e4de6a542e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jhpwq\" (UID: \"00966140-d092-4cd8-8794-f5e4de6a542e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"
Sep 09 06:56:01.557549 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.557509    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qqkzl\" (UniqueName: \"kubernetes.io/projected/00966140-d092-4cd8-8794-f5e4de6a542e-kube-api-access-qqkzl\") pod \"cert-manager-cainjector-559cdd8c87-jhpwq\" (UID: \"00966140-d092-4cd8-8794-f5e4de6a542e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"
Sep 09 06:56:01.557549 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.557555    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00966140-d092-4cd8-8794-f5e4de6a542e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jhpwq\" (UID: \"00966140-d092-4cd8-8794-f5e4de6a542e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"
Sep 09 06:56:01.566617 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.566573    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/00966140-d092-4cd8-8794-f5e4de6a542e-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jhpwq\" (UID: \"00966140-d092-4cd8-8794-f5e4de6a542e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"
Sep 09 06:56:01.566738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.566679    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqkzl\" (UniqueName: \"kubernetes.io/projected/00966140-d092-4cd8-8794-f5e4de6a542e-kube-api-access-qqkzl\") pod \"cert-manager-cainjector-559cdd8c87-jhpwq\" (UID: \"00966140-d092-4cd8-8794-f5e4de6a542e\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"
Sep 09 06:56:01.650569 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.650479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"
Sep 09 06:56:01.926730 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:01.926701    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq"]
Sep 09 06:56:01.929487 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:56:01.929460    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00966140_d092_4cd8_8794_f5e4de6a542e.slice/crio-7bbaad127c09e4155a9477fa60b147afd13f62ef9222ea939789e345ebbb8bfb WatchSource:0}: Error finding container 7bbaad127c09e4155a9477fa60b147afd13f62ef9222ea939789e345ebbb8bfb: Status 404 returned error can't find the container with id 7bbaad127c09e4155a9477fa60b147afd13f62ef9222ea939789e345ebbb8bfb
Sep 09 06:56:02.405253 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:02.405160    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq" event={"ID":"00966140-d092-4cd8-8794-f5e4de6a542e","Type":"ContainerStarted","Data":"7bbaad127c09e4155a9477fa60b147afd13f62ef9222ea939789e345ebbb8bfb"}
Sep 09 06:56:03.857983 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:03.857938    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:03.957911 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:03.957874    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:05.419744 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:05.419659    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq" event={"ID":"00966140-d092-4cd8-8794-f5e4de6a542e","Type":"ContainerStarted","Data":"c9fe4ce7271a1ca15318d079416f37d2d70350de36de68efe710a98473c07bcc"}
Sep 09 06:56:05.439064 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:05.439003    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jhpwq" podStartSLOduration=1.22877259 podStartE2EDuration="4.438984885s" totalImagesPullingTime="3.210212295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:56:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:56:05.435781949 +0000 UTC m=+297.806041035" watchObservedRunningTime="2026-09-09 06:56:05.438984885 +0000 UTC m=+297.809243974"
Sep 09 06:56:06.028197 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:06.028163    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:06.179773 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:06.179739    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:08.056582 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:08.056552    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 06:56:08.061917 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:08.061889    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 06:56:08.074886 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:08.074859    2563 kubelet.go:1648] "Image garbage collection succeeded"
Sep 09 06:56:08.557342 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:08.557305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:11.575225 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:11.575183    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:11.671668 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:11.671636    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:17.749324 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:17.749284    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:17.870606 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:17.870552    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:19.439354 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:19.439322    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:33.672834 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:33.672801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:33.766134 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:33.766092    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:35.509527 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:35.509492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:35.613308 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:35.613276    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:36.209831 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.209780    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:36.565539 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.565455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:36.698064 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.698027    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"]
Sep 09 06:56:36.701232 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.701216    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.704484 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.704464    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 09 06:56:36.705070 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.705048    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 09 06:56:36.705180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.705128    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 09 06:56:36.705180 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.705146    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 09 06:56:36.705304 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.705245    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-q96d5\""
Sep 09 06:56:36.731751 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.731724    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"]
Sep 09 06:56:36.758118 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.758084    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b72de59-473b-4f4a-997f-b5e950d1a3ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.758266 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.758128    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b72de59-473b-4f4a-997f-b5e950d1a3ef-manifests\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.758266 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.758151    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b72de59-473b-4f4a-997f-b5e950d1a3ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.758266 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.758249    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b72de59-473b-4f4a-997f-b5e950d1a3ef-tmp\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.758375 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.758314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv5gx\" (UniqueName: \"kubernetes.io/projected/7b72de59-473b-4f4a-997f-b5e950d1a3ef-kube-api-access-fv5gx\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.859498 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.859408    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b72de59-473b-4f4a-997f-b5e950d1a3ef-tmp\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.859498 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.859467    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv5gx\" (UniqueName: \"kubernetes.io/projected/7b72de59-473b-4f4a-997f-b5e950d1a3ef-kube-api-access-fv5gx\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.859713 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.859581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b72de59-473b-4f4a-997f-b5e950d1a3ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.859713 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.859663    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b72de59-473b-4f4a-997f-b5e950d1a3ef-manifests\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.859713 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.859704    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b72de59-473b-4f4a-997f-b5e950d1a3ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.859997 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.859972    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7b72de59-473b-4f4a-997f-b5e950d1a3ef-manifests\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.861925 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.861890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7b72de59-473b-4f4a-997f-b5e950d1a3ef-tmp\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.862042 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.861985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7b72de59-473b-4f4a-997f-b5e950d1a3ef-webhook-cert\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.862042 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.862009    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7b72de59-473b-4f4a-997f-b5e950d1a3ef-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.880042 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.880016    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv5gx\" (UniqueName: \"kubernetes.io/projected/7b72de59-473b-4f4a-997f-b5e950d1a3ef-kube-api-access-fv5gx\") pod \"opendatahub-operator-controller-manager-7c69cbcb-22cv5\" (UID: \"7b72de59-473b-4f4a-997f-b5e950d1a3ef\") " pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:36.935738 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:36.935707    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:37.011063 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:37.011026    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:37.093765 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:37.093735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:37.237596 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:37.237564    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:37.295216 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:37.295188    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"]
Sep 09 06:56:37.299355 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:56:37.297336    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b72de59_473b_4f4a_997f_b5e950d1a3ef.slice/crio-64350b3c368fd34dcb7d1d794f6ed3d7e58cac7387dace2e4016dc964613fa00 WatchSource:0}: Error finding container 64350b3c368fd34dcb7d1d794f6ed3d7e58cac7387dace2e4016dc964613fa00: Status 404 returned error can't find the container with id 64350b3c368fd34dcb7d1d794f6ed3d7e58cac7387dace2e4016dc964613fa00
Sep 09 06:56:37.300372 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:37.300354    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 06:56:37.338999 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:37.338967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:37.499781 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:37.499694    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:37.542185 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:37.542159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5" event={"ID":"7b72de59-473b-4f4a-997f-b5e950d1a3ef","Type":"ContainerStarted","Data":"64350b3c368fd34dcb7d1d794f6ed3d7e58cac7387dace2e4016dc964613fa00"}
Sep 09 06:56:41.564604 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:41.564559    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5" event={"ID":"7b72de59-473b-4f4a-997f-b5e950d1a3ef","Type":"ContainerStarted","Data":"0a47dd8296e772b047cbd7aa6f9e0013ecb0dc3ea0ed5baaea1cc9bca415806f"}
Sep 09 06:56:42.570136 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:42.570095    2563 generic.go:356] "Generic (PLEG): container finished" podID="7b72de59-473b-4f4a-997f-b5e950d1a3ef" containerID="0a47dd8296e772b047cbd7aa6f9e0013ecb0dc3ea0ed5baaea1cc9bca415806f" exitCode=0
Sep 09 06:56:42.570532 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:42.570151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5" event={"ID":"7b72de59-473b-4f4a-997f-b5e950d1a3ef","Type":"ContainerDied","Data":"0a47dd8296e772b047cbd7aa6f9e0013ecb0dc3ea0ed5baaea1cc9bca415806f"}
Sep 09 06:56:43.576449 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:43.576415    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5" event={"ID":"7b72de59-473b-4f4a-997f-b5e950d1a3ef","Type":"ContainerStarted","Data":"d57dd072a1c02638893404db000dd1d97f32bb32df0f4e0824e1d05225abedeb"}
Sep 09 06:56:43.576864 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:43.576527    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:43.599927 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:43.599883    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5" podStartSLOduration=2.8376470769999997 podStartE2EDuration="7.599868469s" totalImagesPullingTime="3.734128056s" totalInitContainerRuntime="1.028093336s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:56:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:56:43.598297925 +0000 UTC m=+335.968557015" watchObservedRunningTime="2026-09-09 06:56:43.599868469 +0000 UTC m=+335.970127556"
Sep 09 06:56:54.583220 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:54.583188    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7c69cbcb-22cv5"
Sep 09 06:56:56.638262 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:56.638226    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:56.786615 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:56.786558    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:57.044443 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.044414    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:57.115247 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.115212    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"]
Sep 09 06:56:57.126791 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.126769    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.129747 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.129722    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 09 06:56:57.130831 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.130812    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 09 06:56:57.141700 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.141679    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"]
Sep 09 06:56:57.235032 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.234992    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8r8z\" (UniqueName: \"kubernetes.io/projected/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-kube-api-access-t8r8z\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.235204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.235060    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tls-certs\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.235204 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.235117    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tmp\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.336211 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.336117    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8r8z\" (UniqueName: \"kubernetes.io/projected/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-kube-api-access-t8r8z\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.336211 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.336201    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tls-certs\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.336448 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.336236    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tmp\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.336448 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:56:57.336331    2563 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Sep 09 06:56:57.336448 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:56:57.336411    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tls-certs podName:0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c nodeName:}" failed. No retries permitted until 2026-09-09 06:56:57.836395675 +0000 UTC m=+350.206654740 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tls-certs") pod "kube-auth-proxy-56d6c696f5-z4nx7" (UID: "0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c") : secret "kube-auth-proxy-tls" not found
Sep 09 06:56:57.338483 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.338457    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tmp\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.345731 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.345706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8r8z\" (UniqueName: \"kubernetes.io/projected/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-kube-api-access-t8r8z\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.769848 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.769816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:57.840200 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.840164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tls-certs\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:57.842653 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:57.842626    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c-tls-certs\") pod \"kube-auth-proxy-56d6c696f5-z4nx7\" (UID: \"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c\") " pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:58.036389 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.036306    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"
Sep 09 06:56:58.325578 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.325498    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7"]
Sep 09 06:56:58.329001 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:56:58.328974    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cc63f11_fa25_45df_a2ba_8d7f2a1a0e5c.slice/crio-257b1e94ddf063cf6a6b78d546a3bff025e9dabd7506bb56df14531fd74c0c17 WatchSource:0}: Error finding container 257b1e94ddf063cf6a6b78d546a3bff025e9dabd7506bb56df14531fd74c0c17: Status 404 returned error can't find the container with id 257b1e94ddf063cf6a6b78d546a3bff025e9dabd7506bb56df14531fd74c0c17
Sep 09 06:56:58.642125 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.642043    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7" event={"ID":"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c","Type":"ContainerStarted","Data":"257b1e94ddf063cf6a6b78d546a3bff025e9dabd7506bb56df14531fd74c0c17"}
Sep 09 06:56:58.670117 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.670089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:58.680157 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.680110    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"]
Sep 09 06:56:58.684503 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.684488    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.687206 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.687185    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 09 06:56:58.687307 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.687212    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 09 06:56:58.687307 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.687262    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 09 06:56:58.687307 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.687280    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-b6v8g\""
Sep 09 06:56:58.687465 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.687268    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 09 06:56:58.697331 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.697308    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"]
Sep 09 06:56:58.765832 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.765785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:56:58.850652 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.850616    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddprv\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-kube-api-access-ddprv\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.850978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.850657    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.850978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.850691    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.850978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.850723    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.850978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.850763    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.850978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.850819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126ec7-cdd9-4776-bea0-c363e7b3554d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.850978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.850847    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.952281 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.952246    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126ec7-cdd9-4776-bea0-c363e7b3554d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.952450 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.952289    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.952450 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.952334    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddprv\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-kube-api-access-ddprv\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.952450 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.952365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.952450 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.952395    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.952450 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.952416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.952742 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.952472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.953109 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.953077    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.954745 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.954719    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.955027 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.955002    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126ec7-cdd9-4776-bea0-c363e7b3554d-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.955131 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.955094    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.955191 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.955130    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.966327 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.966307    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.967709 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.967690    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddprv\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-kube-api-access-ddprv\") pod \"istiod-openshift-gateway-7b7995d5cd-snppv\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:58.993843 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:58.993818    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:56:59.288391 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:59.288306    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"]
Sep 09 06:56:59.292275 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:56:59.292243    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8126ec7_cdd9_4776_bea0_c363e7b3554d.slice/crio-1eeacbd6946088b3b11df8d7a37cef5fdededb73d68e824b03779ffad85659ee WatchSource:0}: Error finding container 1eeacbd6946088b3b11df8d7a37cef5fdededb73d68e824b03779ffad85659ee: Status 404 returned error can't find the container with id 1eeacbd6946088b3b11df8d7a37cef5fdededb73d68e824b03779ffad85659ee
Sep 09 06:56:59.649625 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:59.649489    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv" event={"ID":"e8126ec7-cdd9-4776-bea0-c363e7b3554d","Type":"ContainerStarted","Data":"1eeacbd6946088b3b11df8d7a37cef5fdededb73d68e824b03779ffad85659ee"}
Sep 09 06:56:59.738083 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:56:59.738049    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:01.660617 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:01.660563    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7" event={"ID":"0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c","Type":"ContainerStarted","Data":"d7e168d677d69a4e64aa734026c5f8268d4a409bf43f414d4b073545f78d0cba"}
Sep 09 06:57:01.677730 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:01.677679    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-56d6c696f5-z4nx7" podStartSLOduration=1.9042518259999999 podStartE2EDuration="4.677663672s" totalImagesPullingTime="2.773411846s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:56:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:57:01.676922127 +0000 UTC m=+354.047181239" watchObservedRunningTime="2026-09-09 06:57:01.677663672 +0000 UTC m=+354.047922739"
Sep 09 06:57:03.251465 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:03.251425    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 06:57:03.251760 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:03.251506    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 06:57:03.671566 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:03.671472    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv" event={"ID":"e8126ec7-cdd9-4776-bea0-c363e7b3554d","Type":"ContainerStarted","Data":"e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b"}
Sep 09 06:57:03.671749 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:03.671625    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:57:03.699045 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:03.698991    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv" podStartSLOduration=1.7428167270000001 podStartE2EDuration="5.698972732s" totalImagesPullingTime="3.956156005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:56:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:57:03.697847599 +0000 UTC m=+356.068106688" watchObservedRunningTime="2026-09-09 06:57:03.698972732 +0000 UTC m=+356.069231839"
Sep 09 06:57:04.347758 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:04.347723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:04.444417 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:04.444379    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:04.678025 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:04.677989    2563 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-snppv container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 06:57:04.678188 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:04.678052    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv" podUID="e8126ec7-cdd9-4776-bea0-c363e7b3554d" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 06:57:05.106529 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:05.106455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:05.223097 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:05.223068    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:05.323479 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:05.323445    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:07.677031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:07.677001    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:57:08.111258 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.111175    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:08.247651 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.247611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:08.350064 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.350027    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:08.445792 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.445763    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:08.545340 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.545303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:08.866948 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.866873    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s66sj"]
Sep 09 06:57:08.868228 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.868199    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:08.871710 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.871692    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:08.875186 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.875169    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-k5pkh\""
Sep 09 06:57:08.875690 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.875676    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Sep 09 06:57:08.883690 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.883668    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s66sj"]
Sep 09 06:57:08.993011 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:08.992977    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:09.035434 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.035382    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c696554e-ecfa-48ae-8f88-d5e86f4f331c-tls-certs\") pod \"model-serving-api-54d89bd5fc-s66sj\" (UID: \"c696554e-ecfa-48ae-8f88-d5e86f4f331c\") " pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:09.035609 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.035449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl5v7\" (UniqueName: \"kubernetes.io/projected/c696554e-ecfa-48ae-8f88-d5e86f4f331c-kube-api-access-sl5v7\") pod \"model-serving-api-54d89bd5fc-s66sj\" (UID: \"c696554e-ecfa-48ae-8f88-d5e86f4f331c\") " pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:09.136097 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.136002    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c696554e-ecfa-48ae-8f88-d5e86f4f331c-tls-certs\") pod \"model-serving-api-54d89bd5fc-s66sj\" (UID: \"c696554e-ecfa-48ae-8f88-d5e86f4f331c\") " pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:09.136097 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.136056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl5v7\" (UniqueName: \"kubernetes.io/projected/c696554e-ecfa-48ae-8f88-d5e86f4f331c-kube-api-access-sl5v7\") pod \"model-serving-api-54d89bd5fc-s66sj\" (UID: \"c696554e-ecfa-48ae-8f88-d5e86f4f331c\") " pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:09.138492 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.138466    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c696554e-ecfa-48ae-8f88-d5e86f4f331c-tls-certs\") pod \"model-serving-api-54d89bd5fc-s66sj\" (UID: \"c696554e-ecfa-48ae-8f88-d5e86f4f331c\") " pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:09.143910 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.143883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl5v7\" (UniqueName: \"kubernetes.io/projected/c696554e-ecfa-48ae-8f88-d5e86f4f331c-kube-api-access-sl5v7\") pod \"model-serving-api-54d89bd5fc-s66sj\" (UID: \"c696554e-ecfa-48ae-8f88-d5e86f4f331c\") " pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:09.181641 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.181607    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:09.454046 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.454010    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-s66sj"]
Sep 09 06:57:09.457417 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:57:09.457390    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc696554e_ecfa_48ae_8f88_d5e86f4f331c.slice/crio-b2c9210a4f1f5f69c6f1c7d134154e5571f209e92a70cfeb52ab5230ba71d34f WatchSource:0}: Error finding container b2c9210a4f1f5f69c6f1c7d134154e5571f209e92a70cfeb52ab5230ba71d34f: Status 404 returned error can't find the container with id b2c9210a4f1f5f69c6f1c7d134154e5571f209e92a70cfeb52ab5230ba71d34f
Sep 09 06:57:09.705126 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:09.705032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-s66sj" event={"ID":"c696554e-ecfa-48ae-8f88-d5e86f4f331c","Type":"ContainerStarted","Data":"b2c9210a4f1f5f69c6f1c7d134154e5571f209e92a70cfeb52ab5230ba71d34f"}
Sep 09 06:57:11.716661 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:11.716620    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-s66sj" event={"ID":"c696554e-ecfa-48ae-8f88-d5e86f4f331c","Type":"ContainerStarted","Data":"c3e2971714ed40ba3047d3c4f7fd8079ede41fb7efb7f0672e65c4448bbd290b"}
Sep 09 06:57:11.717112 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:11.716797    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:11.735248 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:11.735199    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-s66sj" podStartSLOduration=2.19196766 podStartE2EDuration="3.735183983s" totalImagesPullingTime="1.543216323s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:57:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:57:11.733396636 +0000 UTC m=+364.103655726" watchObservedRunningTime="2026-09-09 06:57:11.735183983 +0000 UTC m=+364.105443071"
Sep 09 06:57:15.429698 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.429649    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"]
Sep 09 06:57:15.433279 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.433260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.460214 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.460186    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"]
Sep 09 06:57:15.490991 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.490959    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvhh4\" (UniqueName: \"kubernetes.io/projected/73a2873f-7174-4daf-8599-c935caa030c6-kube-api-access-tvhh4\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.491135 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.490995    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a2873f-7174-4daf-8599-c935caa030c6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.491135 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.491020    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.491135 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.491064    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.491135 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.491100    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.491541 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.491153    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73a2873f-7174-4daf-8599-c935caa030c6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.491541 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.491195    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a2873f-7174-4daf-8599-c935caa030c6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.591883 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.591838    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.592067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.591891    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73a2873f-7174-4daf-8599-c935caa030c6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.592067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.591925    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a2873f-7174-4daf-8599-c935caa030c6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.592067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.591993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvhh4\" (UniqueName: \"kubernetes.io/projected/73a2873f-7174-4daf-8599-c935caa030c6-kube-api-access-tvhh4\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.592067 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.592030    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a2873f-7174-4daf-8599-c935caa030c6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.592262 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.592070    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.592262 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.592148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.593398 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.593360    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/73a2873f-7174-4daf-8599-c935caa030c6-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.594907 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.594871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.595031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.594958    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/73a2873f-7174-4daf-8599-c935caa030c6-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.595031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.594984    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.595133 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.595080    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/73a2873f-7174-4daf-8599-c935caa030c6-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.609474 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.609442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/73a2873f-7174-4daf-8599-c935caa030c6-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.609615 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.609525    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvhh4\" (UniqueName: \"kubernetes.io/projected/73a2873f-7174-4daf-8599-c935caa030c6-kube-api-access-tvhh4\") pod \"istiod-openshift-gateway-5677ccb49b-9v6l4\" (UID: \"73a2873f-7174-4daf-8599-c935caa030c6\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:15.742457 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:15.742378    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:16.030510 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:16.030483    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"]
Sep 09 06:57:16.031732 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:57:16.031704    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73a2873f_7174_4daf_8599_c935caa030c6.slice/crio-c040463eed0ac42ac89bf892d1dc883ca02ede9330c7207be27e1a32589ce03a WatchSource:0}: Error finding container c040463eed0ac42ac89bf892d1dc883ca02ede9330c7207be27e1a32589ce03a: Status 404 returned error can't find the container with id c040463eed0ac42ac89bf892d1dc883ca02ede9330c7207be27e1a32589ce03a
Sep 09 06:57:16.033893 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:16.033862    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 06:57:16.033982 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:16.033921    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 06:57:16.739874 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:16.739841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4" event={"ID":"73a2873f-7174-4daf-8599-c935caa030c6","Type":"ContainerStarted","Data":"c891e14b2bb0a643da6da79601ebbc65d4bdde95adb29c538cc30d32729852a8"}
Sep 09 06:57:16.740240 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:16.739892    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4" event={"ID":"73a2873f-7174-4daf-8599-c935caa030c6","Type":"ContainerStarted","Data":"c040463eed0ac42ac89bf892d1dc883ca02ede9330c7207be27e1a32589ce03a"}
Sep 09 06:57:16.740240 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:16.739949    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:16.771235 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:16.771186    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4" podStartSLOduration=1.771171339 podStartE2EDuration="1.771171339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:57:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:57:16.770936811 +0000 UTC m=+369.141195899" watchObservedRunningTime="2026-09-09 06:57:16.771171339 +0000 UTC m=+369.141430427"
Sep 09 06:57:17.746831 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:17.746803    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-9v6l4"
Sep 09 06:57:17.838035 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:17.838004    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"]
Sep 09 06:57:17.838220 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:17.838201    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv" podUID="e8126ec7-cdd9-4776-bea0-c363e7b3554d" containerName="discovery" containerID="cri-o://e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b" gracePeriod=30
Sep 09 06:57:18.083322 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.083295    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:57:18.117546 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.117512    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-kubeconfig\") pod \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") "
Sep 09 06:57:18.117727 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.117558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddprv\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-kube-api-access-ddprv\") pod \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") "
Sep 09 06:57:18.117727 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.117632    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126ec7-cdd9-4776-bea0-c363e7b3554d-local-certs\") pod \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") "
Sep 09 06:57:18.117727 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.117662    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-token\") pod \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") "
Sep 09 06:57:18.117727 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.117707    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-cacerts\") pod \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") "
Sep 09 06:57:18.117943 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.117744    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-dns-cert\") pod \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") "
Sep 09 06:57:18.117943 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.117771    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-ca-configmap\") pod \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\" (UID: \"e8126ec7-cdd9-4776-bea0-c363e7b3554d\") "
Sep 09 06:57:18.118413 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.118374    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e8126ec7-cdd9-4776-bea0-c363e7b3554d" (UID: "e8126ec7-cdd9-4776-bea0-c363e7b3554d"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:57:18.121313 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.121260    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8126ec7-cdd9-4776-bea0-c363e7b3554d-local-certs" (OuterVolumeSpecName: "local-certs") pod "e8126ec7-cdd9-4776-bea0-c363e7b3554d" (UID: "e8126ec7-cdd9-4776-bea0-c363e7b3554d"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 06:57:18.121478 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.121434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e8126ec7-cdd9-4776-bea0-c363e7b3554d" (UID: "e8126ec7-cdd9-4776-bea0-c363e7b3554d"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:57:18.121631 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.121547    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-kube-api-access-ddprv" (OuterVolumeSpecName: "kube-api-access-ddprv") pod "e8126ec7-cdd9-4776-bea0-c363e7b3554d" (UID: "e8126ec7-cdd9-4776-bea0-c363e7b3554d"). InnerVolumeSpecName "kube-api-access-ddprv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 06:57:18.121973 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.121824    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-cacerts" (OuterVolumeSpecName: "cacerts") pod "e8126ec7-cdd9-4776-bea0-c363e7b3554d" (UID: "e8126ec7-cdd9-4776-bea0-c363e7b3554d"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:57:18.121973 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.121843    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e8126ec7-cdd9-4776-bea0-c363e7b3554d" (UID: "e8126ec7-cdd9-4776-bea0-c363e7b3554d"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:57:18.122219 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.122191    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-token" (OuterVolumeSpecName: "istio-token") pod "e8126ec7-cdd9-4776-bea0-c363e7b3554d" (UID: "e8126ec7-cdd9-4776-bea0-c363e7b3554d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 06:57:18.184695 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.184662    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:18.218830 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.218801    2563 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-kubeconfig\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:57:18.218830 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.218828    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddprv\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-kube-api-access-ddprv\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:57:18.218830 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.218838    2563 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e8126ec7-cdd9-4776-bea0-c363e7b3554d-local-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:57:18.219052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.218846    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:57:18.219052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.218854    2563 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-cacerts\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:57:18.219052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.218862    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-dns-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:57:18.219052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.218870    2563 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e8126ec7-cdd9-4776-bea0-c363e7b3554d-istio-csr-ca-configmap\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:57:18.258166 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.258083    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"]
Sep 09 06:57:18.258533 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.258515    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8126ec7-cdd9-4776-bea0-c363e7b3554d" containerName="discovery"
Sep 09 06:57:18.258624 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.258536    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8126ec7-cdd9-4776-bea0-c363e7b3554d" containerName="discovery"
Sep 09 06:57:18.258690 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.258653    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8126ec7-cdd9-4776-bea0-c363e7b3554d" containerName="discovery"
Sep 09 06:57:18.261637 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.261616    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:18.265284 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.265262    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-sh287\""
Sep 09 06:57:18.265396 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.265310    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 09 06:57:18.280297 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.280266    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"]
Sep 09 06:57:18.306728 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.306699    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:18.320105 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.320078    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xlsxd\" (UniqueName: \"kubernetes.io/projected/25820297-c442-4e17-b230-faa402080612-kube-api-access-xlsxd\") pod \"kserve-controller-manager-d6f4b65b8-c5txd\" (UID: \"25820297-c442-4e17-b230-faa402080612\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:18.320263 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.320135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5txd\" (UID: \"25820297-c442-4e17-b230-faa402080612\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:18.420945 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.420904    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5txd\" (UID: \"25820297-c442-4e17-b230-faa402080612\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:18.421127 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.421025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xlsxd\" (UniqueName: \"kubernetes.io/projected/25820297-c442-4e17-b230-faa402080612-kube-api-access-xlsxd\") pod \"kserve-controller-manager-d6f4b65b8-c5txd\" (UID: \"25820297-c442-4e17-b230-faa402080612\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:18.421127 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:57:18.421057    2563 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 09 06:57:18.421127 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:57:18.421126    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert podName:25820297-c442-4e17-b230-faa402080612 nodeName:}" failed. No retries permitted until 2026-09-09 06:57:18.92110633 +0000 UTC m=+371.291365418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert") pod "kserve-controller-manager-d6f4b65b8-c5txd" (UID: "25820297-c442-4e17-b230-faa402080612") : secret "kserve-webhook-server-cert" not found
Sep 09 06:57:18.436638 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.436606    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xlsxd\" (UniqueName: \"kubernetes.io/projected/25820297-c442-4e17-b230-faa402080612-kube-api-access-xlsxd\") pod \"kserve-controller-manager-d6f4b65b8-c5txd\" (UID: \"25820297-c442-4e17-b230-faa402080612\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:18.540434 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.540337    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:18.750695 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.750660    2563 generic.go:356] "Generic (PLEG): container finished" podID="e8126ec7-cdd9-4776-bea0-c363e7b3554d" containerID="e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b" exitCode=0
Sep 09 06:57:18.751082 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.750722    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"
Sep 09 06:57:18.751082 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.750736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv" event={"ID":"e8126ec7-cdd9-4776-bea0-c363e7b3554d","Type":"ContainerDied","Data":"e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b"}
Sep 09 06:57:18.751082 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.750815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv" event={"ID":"e8126ec7-cdd9-4776-bea0-c363e7b3554d","Type":"ContainerDied","Data":"1eeacbd6946088b3b11df8d7a37cef5fdededb73d68e824b03779ffad85659ee"}
Sep 09 06:57:18.751082 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.750839    2563 scope.go:117] "RemoveContainer" containerID="e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b"
Sep 09 06:57:18.759979 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.759964    2563 scope.go:117] "RemoveContainer" containerID="e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b"
Sep 09 06:57:18.760227 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:57:18.760210    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b\": container with ID starting with e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b not found: ID does not exist" containerID="e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b"
Sep 09 06:57:18.760270 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.760235    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b"} err="failed to get container status \"e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b\": rpc error: code = NotFound desc = could not find container \"e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b\": container with ID starting with e0e4ed4d1c08754346c5d4eda57885d79cd3665eafc38fdb575c4f639498f39b not found: ID does not exist"
Sep 09 06:57:18.779455 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.778878    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"]
Sep 09 06:57:18.783380 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.783354    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-snppv"]
Sep 09 06:57:18.925637 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:18.925579    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5txd\" (UID: \"25820297-c442-4e17-b230-faa402080612\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:18.925796 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:57:18.925725    2563 secret.go:189] Couldn't get secret opendatahub/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found
Sep 09 06:57:18.925796 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:57:18.925789    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert podName:25820297-c442-4e17-b230-faa402080612 nodeName:}" failed. No retries permitted until 2026-09-09 06:57:19.925774129 +0000 UTC m=+372.296033200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert") pod "kserve-controller-manager-d6f4b65b8-c5txd" (UID: "25820297-c442-4e17-b230-faa402080612") : secret "kserve-webhook-server-cert" not found
Sep 09 06:57:19.004920 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:19.004884    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:19.493342 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:19.493309    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:19.598546 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:19.598488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:19.764169 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:19.764073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:19.936624 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:19.936567    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5txd\" (UID: \"25820297-c442-4e17-b230-faa402080612\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:19.938928 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:19.938909    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/25820297-c442-4e17-b230-faa402080612-cert\") pod \"kserve-controller-manager-d6f4b65b8-c5txd\" (UID: \"25820297-c442-4e17-b230-faa402080612\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:20.072075 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:20.071978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:20.116340 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:20.116304    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8126ec7-cdd9-4776-bea0-c363e7b3554d" path="/var/lib/kubelet/pods/e8126ec7-cdd9-4776-bea0-c363e7b3554d/volumes"
Sep 09 06:57:20.374410 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:20.374380    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"]
Sep 09 06:57:20.375677 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:57:20.375647    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25820297_c442_4e17_b230_faa402080612.slice/crio-efd97d812af6c954c40776252a17d4228003d1359d403affec3ff72cf4c4ebc0 WatchSource:0}: Error finding container efd97d812af6c954c40776252a17d4228003d1359d403affec3ff72cf4c4ebc0: Status 404 returned error can't find the container with id efd97d812af6c954c40776252a17d4228003d1359d403affec3ff72cf4c4ebc0
Sep 09 06:57:20.761433 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:20.761399    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd" event={"ID":"25820297-c442-4e17-b230-faa402080612","Type":"ContainerStarted","Data":"efd97d812af6c954c40776252a17d4228003d1359d403affec3ff72cf4c4ebc0"}
Sep 09 06:57:22.725649 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:22.725612    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-s66sj"
Sep 09 06:57:23.779050 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:23.779008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd" event={"ID":"25820297-c442-4e17-b230-faa402080612","Type":"ContainerStarted","Data":"9bc8d172ba430dcd417c91fa886c86eb8cb15977afcf192b9a3bd166fe704e84"}
Sep 09 06:57:23.779450 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:23.779113    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:57:23.810455 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:23.810406    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd" podStartSLOduration=3.452486036 podStartE2EDuration="5.810391162s" totalImagesPullingTime="2.357905126s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:57:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:57:23.80845256 +0000 UTC m=+376.178711649" watchObservedRunningTime="2026-09-09 06:57:23.810391162 +0000 UTC m=+376.180650250"
Sep 09 06:57:33.620824 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.620789    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6767db5f76-9ddhv"]
Sep 09 06:57:33.624785 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.624763    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.637568 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.637537    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6767db5f76-9ddhv"]
Sep 09 06:57:33.679215 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.679167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:33.753492 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.753453    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-trusted-ca-bundle\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.753707 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.753511    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-oauth-serving-cert\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.753707 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.753549    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-oauth-config\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.753707 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.753610    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-serving-cert\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.753707 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.753653    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-service-ca\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.753842 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.753722    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxmtz\" (UniqueName: \"kubernetes.io/projected/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-kube-api-access-xxmtz\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.753842 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.753750    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-config\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.854923 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.854890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-serving-cert\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.854923 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.854923    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-service-ca\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.855165 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.854949    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxmtz\" (UniqueName: \"kubernetes.io/projected/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-kube-api-access-xxmtz\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.855165 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.854971    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-config\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.855165 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.855022    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-trusted-ca-bundle\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.855165 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.855057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-oauth-serving-cert\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.855165 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.855079    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-oauth-config\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.855803 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.855778    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-config\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.855803 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.855797    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-service-ca\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.855948 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.855916    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-oauth-serving-cert\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.856002 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.855983    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-trusted-ca-bundle\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.857551 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.857520    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-oauth-config\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.857700 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.857681    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-console-serving-cert\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.863641 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.863620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxmtz\" (UniqueName: \"kubernetes.io/projected/85743ee0-2b6e-4cf2-b85e-2e65fa612b0d-kube-api-access-xxmtz\") pod \"console-6767db5f76-9ddhv\" (UID: \"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d\") " pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:33.934687 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:33.934658    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:34.214026 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:34.213998    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6767db5f76-9ddhv"]
Sep 09 06:57:34.215796 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:57:34.215766    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85743ee0_2b6e_4cf2_b85e_2e65fa612b0d.slice/crio-4987331d15f308e09e6d5f121ce3104db8d101c03dd86c754e3ecf375fb01814 WatchSource:0}: Error finding container 4987331d15f308e09e6d5f121ce3104db8d101c03dd86c754e3ecf375fb01814: Status 404 returned error can't find the container with id 4987331d15f308e09e6d5f121ce3104db8d101c03dd86c754e3ecf375fb01814
Sep 09 06:57:34.830212 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:34.830171    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6767db5f76-9ddhv" event={"ID":"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d","Type":"ContainerStarted","Data":"357a3129c0a8cf6f121a9d192cf2a10b13a9b40d8e0a1989ef2e3a8e6b30dcb8"}
Sep 09 06:57:34.830577 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:34.830230    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6767db5f76-9ddhv" event={"ID":"85743ee0-2b6e-4cf2-b85e-2e65fa612b0d","Type":"ContainerStarted","Data":"4987331d15f308e09e6d5f121ce3104db8d101c03dd86c754e3ecf375fb01814"}
Sep 09 06:57:34.851473 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:34.851422    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6767db5f76-9ddhv" podStartSLOduration=1.8514064989999999 podStartE2EDuration="1.851406499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:57:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:57:34.849892521 +0000 UTC m=+387.220151624" watchObservedRunningTime="2026-09-09 06:57:34.851406499 +0000 UTC m=+387.221665587"
Sep 09 06:57:43.935752 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:43.935717    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:43.936119 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:43.935773    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:43.940313 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:43.940290    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:44.879660 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:44.879633    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6767db5f76-9ddhv"
Sep 09 06:57:44.928883 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:44.928843    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d444f74cd-qt9wr"]
Sep 09 06:57:45.039351 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:45.039301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:45.135914 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:45.135833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:45.192997 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:45.192967    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:57:54.789099 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:57:54.789064    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-c5txd"
Sep 09 06:58:08.872260 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:08.872225    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:08.972491 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:08.972435    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:09.951815 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:09.951779    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d444f74cd-qt9wr" podUID="be3c5945-8f13-4087-aeaf-58594305374c" containerName="console" containerID="cri-o://b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24" gracePeriod=15
Sep 09 06:58:10.192681 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.192656    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:58:10.263368 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263296    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-serving-cert\") pod \"be3c5945-8f13-4087-aeaf-58594305374c\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") "
Sep 09 06:58:10.263368 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263338    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdhbf\" (UniqueName: \"kubernetes.io/projected/be3c5945-8f13-4087-aeaf-58594305374c-kube-api-access-mdhbf\") pod \"be3c5945-8f13-4087-aeaf-58594305374c\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") "
Sep 09 06:58:10.263538 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263369    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-service-ca\") pod \"be3c5945-8f13-4087-aeaf-58594305374c\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") "
Sep 09 06:58:10.263538 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263431    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-trusted-ca-bundle\") pod \"be3c5945-8f13-4087-aeaf-58594305374c\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") "
Sep 09 06:58:10.263538 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263472    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-oauth-serving-cert\") pod \"be3c5945-8f13-4087-aeaf-58594305374c\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") "
Sep 09 06:58:10.263538 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-oauth-config\") pod \"be3c5945-8f13-4087-aeaf-58594305374c\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") "
Sep 09 06:58:10.263744 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263556    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-console-config\") pod \"be3c5945-8f13-4087-aeaf-58594305374c\" (UID: \"be3c5945-8f13-4087-aeaf-58594305374c\") "
Sep 09 06:58:10.263827 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263791    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-service-ca" (OuterVolumeSpecName: "service-ca") pod "be3c5945-8f13-4087-aeaf-58594305374c" (UID: "be3c5945-8f13-4087-aeaf-58594305374c"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:58:10.264122 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.264097    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-console-config" (OuterVolumeSpecName: "console-config") pod "be3c5945-8f13-4087-aeaf-58594305374c" (UID: "be3c5945-8f13-4087-aeaf-58594305374c"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:58:10.264214 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.263876    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "be3c5945-8f13-4087-aeaf-58594305374c" (UID: "be3c5945-8f13-4087-aeaf-58594305374c"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:58:10.264214 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.264128    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "be3c5945-8f13-4087-aeaf-58594305374c" (UID: "be3c5945-8f13-4087-aeaf-58594305374c"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:58:10.265491 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.265467    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "be3c5945-8f13-4087-aeaf-58594305374c" (UID: "be3c5945-8f13-4087-aeaf-58594305374c"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:58:10.265790 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.265771    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "be3c5945-8f13-4087-aeaf-58594305374c" (UID: "be3c5945-8f13-4087-aeaf-58594305374c"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 06:58:10.265851 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.265824    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be3c5945-8f13-4087-aeaf-58594305374c-kube-api-access-mdhbf" (OuterVolumeSpecName: "kube-api-access-mdhbf") pod "be3c5945-8f13-4087-aeaf-58594305374c" (UID: "be3c5945-8f13-4087-aeaf-58594305374c"). InnerVolumeSpecName "kube-api-access-mdhbf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 06:58:10.364704 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.364655    2563 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-trusted-ca-bundle\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:10.364704 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.364692    2563 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-oauth-serving-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:10.364704 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.364701    2563 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-oauth-config\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:10.364704 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.364711    2563 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-console-config\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:10.364704 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.364720    2563 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/be3c5945-8f13-4087-aeaf-58594305374c-console-serving-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:10.364993 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.364729    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdhbf\" (UniqueName: \"kubernetes.io/projected/be3c5945-8f13-4087-aeaf-58594305374c-kube-api-access-mdhbf\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:10.364993 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.364738    2563 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/be3c5945-8f13-4087-aeaf-58594305374c-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:10.993788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.993751    2563 generic.go:356] "Generic (PLEG): container finished" podID="be3c5945-8f13-4087-aeaf-58594305374c" containerID="b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24" exitCode=0
Sep 09 06:58:10.993788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.993790    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d444f74cd-qt9wr" event={"ID":"be3c5945-8f13-4087-aeaf-58594305374c","Type":"ContainerDied","Data":"b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24"}
Sep 09 06:58:10.993788 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.993795    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d444f74cd-qt9wr"
Sep 09 06:58:10.994268 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.993829    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d444f74cd-qt9wr" event={"ID":"be3c5945-8f13-4087-aeaf-58594305374c","Type":"ContainerDied","Data":"25e29b5cb952b6dac7344f7170996cbc5468a65b53ba2c34f312e006917f31d9"}
Sep 09 06:58:10.994268 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:10.993849    2563 scope.go:117] "RemoveContainer" containerID="b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24"
Sep 09 06:58:11.003052 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:11.003032    2563 scope.go:117] "RemoveContainer" containerID="b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24"
Sep 09 06:58:11.003320 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:58:11.003303    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24\": container with ID starting with b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24 not found: ID does not exist" containerID="b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24"
Sep 09 06:58:11.003358 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:11.003329    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24"} err="failed to get container status \"b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24\": rpc error: code = NotFound desc = could not find container \"b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24\": container with ID starting with b3ac4b5f7d5569be21f309ea8578fee784fa79dac858c8cdff8c0b5554b75d24 not found: ID does not exist"
Sep 09 06:58:11.015016 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:11.014993    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d444f74cd-qt9wr"]
Sep 09 06:58:11.019561 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:11.019535    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d444f74cd-qt9wr"]
Sep 09 06:58:12.117170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:12.117133    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be3c5945-8f13-4087-aeaf-58594305374c" path="/var/lib/kubelet/pods/be3c5945-8f13-4087-aeaf-58594305374c/volumes"
Sep 09 06:58:13.309419 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:13.309371    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:25.069963 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:25.069933    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:27.886088 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:27.886055    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:29.212287 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.212254    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"]
Sep 09 06:58:29.212649 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.212624    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be3c5945-8f13-4087-aeaf-58594305374c" containerName="console"
Sep 09 06:58:29.212649 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.212636    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="be3c5945-8f13-4087-aeaf-58594305374c" containerName="console"
Sep 09 06:58:29.212729 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.212689    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="be3c5945-8f13-4087-aeaf-58594305374c" containerName="console"
Sep 09 06:58:29.217177 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.217158    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"
Sep 09 06:58:29.219534 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.219496    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 09 06:58:29.219534 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.219526    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 09 06:58:29.219713 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.219615    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 09 06:58:29.219713 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.219618    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-h4rfj\""
Sep 09 06:58:29.225789 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.225765    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"]
Sep 09 06:58:29.334206 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.334164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q9wbd\" (UniqueName: \"kubernetes.io/projected/9fdaf87c-e8a7-4481-9277-fe6a5dc58c53-kube-api-access-q9wbd\") pod \"dns-operator-controller-manager-7b84cd7c9c-7jp4c\" (UID: \"9fdaf87c-e8a7-4481-9277-fe6a5dc58c53\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"
Sep 09 06:58:29.434925 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.434895    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q9wbd\" (UniqueName: \"kubernetes.io/projected/9fdaf87c-e8a7-4481-9277-fe6a5dc58c53-kube-api-access-q9wbd\") pod \"dns-operator-controller-manager-7b84cd7c9c-7jp4c\" (UID: \"9fdaf87c-e8a7-4481-9277-fe6a5dc58c53\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"
Sep 09 06:58:29.444451 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.444427    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q9wbd\" (UniqueName: \"kubernetes.io/projected/9fdaf87c-e8a7-4481-9277-fe6a5dc58c53-kube-api-access-q9wbd\") pod \"dns-operator-controller-manager-7b84cd7c9c-7jp4c\" (UID: \"9fdaf87c-e8a7-4481-9277-fe6a5dc58c53\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"
Sep 09 06:58:29.455714 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.455687    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:29.528464 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.528392    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"
Sep 09 06:58:29.802279 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.802251    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"]
Sep 09 06:58:29.805033 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:58:29.805006    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9fdaf87c_e8a7_4481_9277_fe6a5dc58c53.slice/crio-d5775003b83b726a376fa642c52a32d52e50c6f72b4222b6b82e52be1c6f2997 WatchSource:0}: Error finding container d5775003b83b726a376fa642c52a32d52e50c6f72b4222b6b82e52be1c6f2997: Status 404 returned error can't find the container with id d5775003b83b726a376fa642c52a32d52e50c6f72b4222b6b82e52be1c6f2997
Sep 09 06:58:29.861918 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.861889    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:29.970914 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:29.970886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:30.082678 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:30.082575    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c" event={"ID":"9fdaf87c-e8a7-4481-9277-fe6a5dc58c53","Type":"ContainerStarted","Data":"d5775003b83b726a376fa642c52a32d52e50c6f72b4222b6b82e52be1c6f2997"}
Sep 09 06:58:30.151324 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:30.151288    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:31.526371 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:31.526255    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:31.841571 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:31.841453    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2x89r"]
Sep 09 06:58:31.846445 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:31.846418    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-2x89r"
Sep 09 06:58:31.849471 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:31.849429    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-zbktm\""
Sep 09 06:58:31.863104 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:31.863070    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2x89r"]
Sep 09 06:58:31.958494 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:31.958455    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjht2\" (UniqueName: \"kubernetes.io/projected/fa370629-32c7-49ff-bd39-fc91088bacee-kube-api-access-qjht2\") pod \"authorino-operator-76b478bd78-2x89r\" (UID: \"fa370629-32c7-49ff-bd39-fc91088bacee\") " pod="kuadrant-system/authorino-operator-76b478bd78-2x89r"
Sep 09 06:58:32.059197 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:32.059148    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjht2\" (UniqueName: \"kubernetes.io/projected/fa370629-32c7-49ff-bd39-fc91088bacee-kube-api-access-qjht2\") pod \"authorino-operator-76b478bd78-2x89r\" (UID: \"fa370629-32c7-49ff-bd39-fc91088bacee\") " pod="kuadrant-system/authorino-operator-76b478bd78-2x89r"
Sep 09 06:58:32.068990 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:32.068956    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjht2\" (UniqueName: \"kubernetes.io/projected/fa370629-32c7-49ff-bd39-fc91088bacee-kube-api-access-qjht2\") pod \"authorino-operator-76b478bd78-2x89r\" (UID: \"fa370629-32c7-49ff-bd39-fc91088bacee\") " pod="kuadrant-system/authorino-operator-76b478bd78-2x89r"
Sep 09 06:58:32.166331 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:32.166247    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-2x89r"
Sep 09 06:58:32.468201 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:32.468173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:32.539366 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:32.539341    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-2x89r"]
Sep 09 06:58:32.541373 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:58:32.541350    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa370629_32c7_49ff_bd39_fc91088bacee.slice/crio-1898552d496fad8b9e447fd9b8f6fb83d5e03f8f30d711113c8002c9935b9230 WatchSource:0}: Error finding container 1898552d496fad8b9e447fd9b8f6fb83d5e03f8f30d711113c8002c9935b9230: Status 404 returned error can't find the container with id 1898552d496fad8b9e447fd9b8f6fb83d5e03f8f30d711113c8002c9935b9230
Sep 09 06:58:32.546669 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:32.546641    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:32.870831 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:32.870754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:32.973428 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:32.973397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:33.100863 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:33.100824    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c" event={"ID":"9fdaf87c-e8a7-4481-9277-fe6a5dc58c53","Type":"ContainerStarted","Data":"6a8172740e03b60e0cc8e0ae8023b85339a263e593345020b49628aca07395b2"}
Sep 09 06:58:33.101042 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:33.100890    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"
Sep 09 06:58:33.102119 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:33.102099    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-2x89r" event={"ID":"fa370629-32c7-49ff-bd39-fc91088bacee","Type":"ContainerStarted","Data":"1898552d496fad8b9e447fd9b8f6fb83d5e03f8f30d711113c8002c9935b9230"}
Sep 09 06:58:33.131527 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:33.131431    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c" podStartSLOduration=1.599113446 podStartE2EDuration="4.131413719s" totalImagesPullingTime="2.532300273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:58:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:58:33.129004533 +0000 UTC m=+445.499263612" watchObservedRunningTime="2026-09-09 06:58:33.131413719 +0000 UTC m=+445.501672804"
Sep 09 06:58:33.267969 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:33.267930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:33.370116 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:33.370079    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:34.033457 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:34.033425    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:34.108543 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:34.108506    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-2x89r" event={"ID":"fa370629-32c7-49ff-bd39-fc91088bacee","Type":"ContainerStarted","Data":"fc493d0c6e4a9859e99ad8696423c8d02074554297a4bdd03d181971f1338a8b"}
Sep 09 06:58:34.108723 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:34.108607    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-2x89r"
Sep 09 06:58:34.127033 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:34.126990    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-2x89r" podStartSLOduration=1.773182665 podStartE2EDuration="3.126956277s" totalImagesPullingTime="1.353773612s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:58:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:58:34.126246217 +0000 UTC m=+446.496505299" watchObservedRunningTime="2026-09-09 06:58:34.126956277 +0000 UTC m=+446.497215364"
Sep 09 06:58:34.129896 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:34.129869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:37.688617 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:37.688569    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:37.790254 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:37.790218    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:40.472973 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:40.472941    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:40.533548 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:40.533502    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:43.726902 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:43.726867    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:43.885480 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:43.885447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:44.116319 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:44.116243    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-7jp4c"
Sep 09 06:58:45.116194 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:45.116158    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-2x89r"
Sep 09 06:58:45.534300 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:45.534246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:45.638031 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:45.637999    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:50.214452 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:50.214415    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:50.315650 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:50.315614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:50.377661 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:50.377627    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:54.118663 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.118633    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"]
Sep 09 06:58:54.122196 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.122180    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:54.124495 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.124481    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gfwrd\""
Sep 09 06:58:54.138154 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.138130    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"]
Sep 09 06:58:54.149037 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.149015    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29f87566-5961-41b1-bf3d-6e75a57cd862-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" (UID: \"29f87566-5961-41b1-bf3d-6e75a57cd862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:54.149170 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.149063    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7p42d\" (UniqueName: \"kubernetes.io/projected/29f87566-5961-41b1-bf3d-6e75a57cd862-kube-api-access-7p42d\") pod \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" (UID: \"29f87566-5961-41b1-bf3d-6e75a57cd862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:54.250504 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.250472    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29f87566-5961-41b1-bf3d-6e75a57cd862-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" (UID: \"29f87566-5961-41b1-bf3d-6e75a57cd862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:54.250699 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.250532    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7p42d\" (UniqueName: \"kubernetes.io/projected/29f87566-5961-41b1-bf3d-6e75a57cd862-kube-api-access-7p42d\") pod \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" (UID: \"29f87566-5961-41b1-bf3d-6e75a57cd862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:54.250902 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.250883    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29f87566-5961-41b1-bf3d-6e75a57cd862-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" (UID: \"29f87566-5961-41b1-bf3d-6e75a57cd862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:54.262254 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.262228    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7p42d\" (UniqueName: \"kubernetes.io/projected/29f87566-5961-41b1-bf3d-6e75a57cd862-kube-api-access-7p42d\") pod \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" (UID: \"29f87566-5961-41b1-bf3d-6e75a57cd862\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:54.432036 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.432010    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:54.709677 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.709652    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"]
Sep 09 06:58:54.711203 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:58:54.711176    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29f87566_5961_41b1_bf3d_6e75a57cd862.slice/crio-896b5e36807042ab6fc7124949269654ab12d2ed867408f0452c78c7a6a8895f WatchSource:0}: Error finding container 896b5e36807042ab6fc7124949269654ab12d2ed867408f0452c78c7a6a8895f: Status 404 returned error can't find the container with id 896b5e36807042ab6fc7124949269654ab12d2ed867408f0452c78c7a6a8895f
Sep 09 06:58:54.766317 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.766283    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"]
Sep 09 06:58:54.771465 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.771437    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"]
Sep 09 06:58:54.805129 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.805091    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"]
Sep 09 06:58:54.810087 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.810065    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:54.830567 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.830539    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"]
Sep 09 06:58:54.855875 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.855846    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kxd5\" (UniqueName: \"kubernetes.io/projected/6fcde9f7-3f0a-450a-b197-6ba830b96f93-kube-api-access-6kxd5\") pod \"kuadrant-operator-controller-manager-5f685c496f-v8j7n\" (UID: \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:54.856018 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.855900    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fcde9f7-3f0a-450a-b197-6ba830b96f93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-v8j7n\" (UID: \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:54.957124 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.957091    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kxd5\" (UniqueName: \"kubernetes.io/projected/6fcde9f7-3f0a-450a-b197-6ba830b96f93-kube-api-access-6kxd5\") pod \"kuadrant-operator-controller-manager-5f685c496f-v8j7n\" (UID: \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:54.957290 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.957137    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fcde9f7-3f0a-450a-b197-6ba830b96f93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-v8j7n\" (UID: \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:54.957472 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.957456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fcde9f7-3f0a-450a-b197-6ba830b96f93-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f685c496f-v8j7n\" (UID: \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:54.968816 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:54.968745    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kxd5\" (UniqueName: \"kubernetes.io/projected/6fcde9f7-3f0a-450a-b197-6ba830b96f93-kube-api-access-6kxd5\") pod \"kuadrant-operator-controller-manager-5f685c496f-v8j7n\" (UID: \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:55.124448 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:55.124415    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:55.414639 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:55.414615    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"]
Sep 09 06:58:55.417334 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:58:55.417299    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fcde9f7_3f0a_450a_b197_6ba830b96f93.slice/crio-53ae6da1d3acb4f9ff4450088a7579b9af57832beb387548dd4d19c5e33ee56b WatchSource:0}: Error finding container 53ae6da1d3acb4f9ff4450088a7579b9af57832beb387548dd4d19c5e33ee56b: Status 404 returned error can't find the container with id 53ae6da1d3acb4f9ff4450088a7579b9af57832beb387548dd4d19c5e33ee56b
Sep 09 06:58:56.218695 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:56.218655    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n" event={"ID":"6fcde9f7-3f0a-450a-b197-6ba830b96f93","Type":"ContainerStarted","Data":"53ae6da1d3acb4f9ff4450088a7579b9af57832beb387548dd4d19c5e33ee56b"}
Sep 09 06:58:56.969375 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:56.969335    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:57.066999 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:57.066956    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:59.238750 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.238710    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n" event={"ID":"6fcde9f7-3f0a-450a-b197-6ba830b96f93","Type":"ContainerStarted","Data":"873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af"}
Sep 09 06:58:59.239203 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.238862    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:58:59.240468 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.240439    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" containerName="manager" containerID="cri-o://8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb" gracePeriod=2
Sep 09 06:58:59.260171 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.260126    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n" podStartSLOduration=2.280367112 podStartE2EDuration="5.260109458s" totalImagesPullingTime="2.979742346s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:58:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:58:59.259096616 +0000 UTC m=+471.629355713" watchObservedRunningTime="2026-09-09 06:58:59.260109458 +0000 UTC m=+471.630368540"
Sep 09 06:58:59.263225 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:58:59.263194    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" is forbidden: User \"system:node:ip-10-0-128-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-206.ec2.internal' and this object" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:59.482613 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.482575    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:59.484685 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:58:59.484660    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" is forbidden: User \"system:node:ip-10-0-128-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-206.ec2.internal' and this object" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:58:59.500794 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.500743    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7p42d\" (UniqueName: \"kubernetes.io/projected/29f87566-5961-41b1-bf3d-6e75a57cd862-kube-api-access-7p42d\") pod \"29f87566-5961-41b1-bf3d-6e75a57cd862\" (UID: \"29f87566-5961-41b1-bf3d-6e75a57cd862\") "
Sep 09 06:58:59.500879 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.500815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29f87566-5961-41b1-bf3d-6e75a57cd862-extensions-socket-volume\") pod \"29f87566-5961-41b1-bf3d-6e75a57cd862\" (UID: \"29f87566-5961-41b1-bf3d-6e75a57cd862\") "
Sep 09 06:58:59.501074 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.501053    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29f87566-5961-41b1-bf3d-6e75a57cd862-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "29f87566-5961-41b1-bf3d-6e75a57cd862" (UID: "29f87566-5961-41b1-bf3d-6e75a57cd862"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 06:58:59.502981 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.502955    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29f87566-5961-41b1-bf3d-6e75a57cd862-kube-api-access-7p42d" (OuterVolumeSpecName: "kube-api-access-7p42d") pod "29f87566-5961-41b1-bf3d-6e75a57cd862" (UID: "29f87566-5961-41b1-bf3d-6e75a57cd862"). InnerVolumeSpecName "kube-api-access-7p42d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 06:58:59.602026 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.601997    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29f87566-5961-41b1-bf3d-6e75a57cd862-extensions-socket-volume\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:59.602026 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.602025    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7p42d\" (UniqueName: \"kubernetes.io/projected/29f87566-5961-41b1-bf3d-6e75a57cd862-kube-api-access-7p42d\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:58:59.699375 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.699344    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:58:59.798890 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:58:59.798809    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:00.117843 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:00.117766    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" path="/var/lib/kubelet/pods/29f87566-5961-41b1-bf3d-6e75a57cd862/volumes"
Sep 09 06:59:00.246674 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:00.246638    2563 generic.go:356] "Generic (PLEG): container finished" podID="29f87566-5961-41b1-bf3d-6e75a57cd862" containerID="8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb" exitCode=2
Sep 09 06:59:00.247101 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:00.246688    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:59:00.247101 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:00.246738    2563 scope.go:117] "RemoveContainer" containerID="8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb"
Sep 09 06:59:00.249229 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:59:00.249203    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" is forbidden: User \"system:node:ip-10-0-128-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-206.ec2.internal' and this object" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:59:00.251061 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:59:00.251032    2563 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5f685c496f-sdxjn\" is forbidden: User \"system:node:ip-10-0-128-206.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-128-206.ec2.internal' and this object" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-sdxjn"
Sep 09 06:59:00.255249 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:00.255233    2563 scope.go:117] "RemoveContainer" containerID="8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb"
Sep 09 06:59:00.255517 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:59:00.255496    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb\": container with ID starting with 8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb not found: ID does not exist" containerID="8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb"
Sep 09 06:59:00.255563 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:00.255526    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb"} err="failed to get container status \"8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb\": rpc error: code = NotFound desc = could not find container \"8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb\": container with ID starting with 8e300794b2f91b11013b57eee0b3d56c8996cfe4149c51f8be9d897e735a13cb not found: ID does not exist"
Sep 09 06:59:04.536270 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:04.536230    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:04.607325 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:04.607293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:04.685986 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:04.685956    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:04.755272 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:04.755233    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:09.683888 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:09.683843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:09.782743 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:09.782710    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:09.882799 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:09.882760    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:10.249518 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:10.249487    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:59:19.678352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:19.678310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:22.782441 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:22.782403    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"]
Sep 09 06:59:22.782926 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:22.782846    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n" podUID="6fcde9f7-3f0a-450a-b197-6ba830b96f93" containerName="manager" containerID="cri-o://873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af" gracePeriod=10
Sep 09 06:59:23.030412 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.030385    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:59:23.115081 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.114994    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kxd5\" (UniqueName: \"kubernetes.io/projected/6fcde9f7-3f0a-450a-b197-6ba830b96f93-kube-api-access-6kxd5\") pod \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\" (UID: \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\") "
Sep 09 06:59:23.115081 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.115068    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fcde9f7-3f0a-450a-b197-6ba830b96f93-extensions-socket-volume\") pod \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\" (UID: \"6fcde9f7-3f0a-450a-b197-6ba830b96f93\") "
Sep 09 06:59:23.115502 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.115462    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6fcde9f7-3f0a-450a-b197-6ba830b96f93-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "6fcde9f7-3f0a-450a-b197-6ba830b96f93" (UID: "6fcde9f7-3f0a-450a-b197-6ba830b96f93"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 06:59:23.117243 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.117223    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fcde9f7-3f0a-450a-b197-6ba830b96f93-kube-api-access-6kxd5" (OuterVolumeSpecName: "kube-api-access-6kxd5") pod "6fcde9f7-3f0a-450a-b197-6ba830b96f93" (UID: "6fcde9f7-3f0a-450a-b197-6ba830b96f93"). InnerVolumeSpecName "kube-api-access-6kxd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 06:59:23.215639 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.215603    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kxd5\" (UniqueName: \"kubernetes.io/projected/6fcde9f7-3f0a-450a-b197-6ba830b96f93-kube-api-access-6kxd5\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:59:23.215639 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.215628    2563 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/6fcde9f7-3f0a-450a-b197-6ba830b96f93-extensions-socket-volume\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:59:23.361044 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.361002    2563 generic.go:356] "Generic (PLEG): container finished" podID="6fcde9f7-3f0a-450a-b197-6ba830b96f93" containerID="873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af" exitCode=0
Sep 09 06:59:23.361044 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.361051    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"
Sep 09 06:59:23.361262 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.361089    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n" event={"ID":"6fcde9f7-3f0a-450a-b197-6ba830b96f93","Type":"ContainerDied","Data":"873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af"}
Sep 09 06:59:23.361262 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.361144    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n" event={"ID":"6fcde9f7-3f0a-450a-b197-6ba830b96f93","Type":"ContainerDied","Data":"53ae6da1d3acb4f9ff4450088a7579b9af57832beb387548dd4d19c5e33ee56b"}
Sep 09 06:59:23.361262 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.361167    2563 scope.go:117] "RemoveContainer" containerID="873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af"
Sep 09 06:59:23.370074 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.370059    2563 scope.go:117] "RemoveContainer" containerID="873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af"
Sep 09 06:59:23.370347 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:59:23.370329    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af\": container with ID starting with 873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af not found: ID does not exist" containerID="873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af"
Sep 09 06:59:23.370393 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.370354    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af"} err="failed to get container status \"873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af\": rpc error: code = NotFound desc = could not find container \"873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af\": container with ID starting with 873131f163838f8ed40ebbe2f0801b9e278f036679a762d141976a21a48e51af not found: ID does not exist"
Sep 09 06:59:23.385458 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.385428    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"]
Sep 09 06:59:23.389761 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:23.389737    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f685c496f-v8j7n"]
Sep 09 06:59:24.116706 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:24.116672    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fcde9f7-3f0a-450a-b197-6ba830b96f93" path="/var/lib/kubelet/pods/6fcde9f7-3f0a-450a-b197-6ba830b96f93/volumes"
Sep 09 06:59:26.731978 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:26.731947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:26.835313 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:26.835274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:28.221714 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:28.221678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:41.245280 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.245245    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:41.249105 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.249078    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jsfxx"]
Sep 09 06:59:41.249449 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.249437    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fcde9f7-3f0a-450a-b197-6ba830b96f93" containerName="manager"
Sep 09 06:59:41.249491 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.249451    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fcde9f7-3f0a-450a-b197-6ba830b96f93" containerName="manager"
Sep 09 06:59:41.249491 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.249463    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" containerName="manager"
Sep 09 06:59:41.249491 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.249469    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" containerName="manager"
Sep 09 06:59:41.249580 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.249521    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="29f87566-5961-41b1-bf3d-6e75a57cd862" containerName="manager"
Sep 09 06:59:41.249580 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.249530    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fcde9f7-3f0a-450a-b197-6ba830b96f93" containerName="manager"
Sep 09 06:59:41.251420 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.251401    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:41.253786 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.253767    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ssn7s\""
Sep 09 06:59:41.253885 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.253767    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 09 06:59:41.264051 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.264028    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jsfxx"]
Sep 09 06:59:41.272754 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.272728    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55rx9\" (UniqueName: \"kubernetes.io/projected/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-kube-api-access-55rx9\") pod \"limitador-limitador-8588749977-jsfxx\" (UID: \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\") " pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:41.272900 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.272875    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-config-file\") pod \"limitador-limitador-8588749977-jsfxx\" (UID: \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\") " pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:41.345020 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.344986    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jsfxx"]
Sep 09 06:59:41.354185 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.354156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:41.373555 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.373524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-config-file\") pod \"limitador-limitador-8588749977-jsfxx\" (UID: \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\") " pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:41.373729 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.373576    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55rx9\" (UniqueName: \"kubernetes.io/projected/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-kube-api-access-55rx9\") pod \"limitador-limitador-8588749977-jsfxx\" (UID: \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\") " pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:41.374431 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.374401    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-config-file\") pod \"limitador-limitador-8588749977-jsfxx\" (UID: \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\") " pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:41.381281 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.381247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55rx9\" (UniqueName: \"kubernetes.io/projected/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-kube-api-access-55rx9\") pod \"limitador-limitador-8588749977-jsfxx\" (UID: \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\") " pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:41.413258 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.413224    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:41.562615 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.562517    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:41.830148 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:41.830047    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jsfxx"]
Sep 09 06:59:41.832783 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:59:41.832754    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d23ded3_822d_4fad_b59d_f0df6ec4beb4.slice/crio-2eb23d4a2cd4c1babb8db1321ba29d8d876c69714dd39bf1448ad636dd91e4e3 WatchSource:0}: Error finding container 2eb23d4a2cd4c1babb8db1321ba29d8d876c69714dd39bf1448ad636dd91e4e3: Status 404 returned error can't find the container with id 2eb23d4a2cd4c1babb8db1321ba29d8d876c69714dd39bf1448ad636dd91e4e3
Sep 09 06:59:42.081476 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:42.081367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:42.168310 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:42.168274    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:42.454679 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:42.454642    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx" event={"ID":"7d23ded3-822d-4fad-b59d-f0df6ec4beb4","Type":"ContainerStarted","Data":"2eb23d4a2cd4c1babb8db1321ba29d8d876c69714dd39bf1448ad636dd91e4e3"}
Sep 09 06:59:43.461290 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:43.461258    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx" event={"ID":"7d23ded3-822d-4fad-b59d-f0df6ec4beb4","Type":"ContainerStarted","Data":"3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab"}
Sep 09 06:59:43.461691 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:43.461320    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:43.480152 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:43.480102    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx" podStartSLOduration=1.236904071 podStartE2EDuration="2.480087085s" totalImagesPullingTime="1.243183014s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:59:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 06:59:43.477940372 +0000 UTC m=+515.848199469" watchObservedRunningTime="2026-09-09 06:59:43.480087085 +0000 UTC m=+515.850346172"
Sep 09 06:59:51.948134 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:51.948097    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:52.006791 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:52.006757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:54.470469 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:54.470433    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:54.635448 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:54.635410    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jsfxx"]
Sep 09 06:59:54.635655 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:54.635610    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx" podUID="7d23ded3-822d-4fad-b59d-f0df6ec4beb4" containerName="limitador" containerID="cri-o://3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab" gracePeriod=30
Sep 09 06:59:55.185435 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.185410    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:55.300213 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.300141    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-config-file\") pod \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\" (UID: \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\") "
Sep 09 06:59:55.300352 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.300227    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55rx9\" (UniqueName: \"kubernetes.io/projected/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-kube-api-access-55rx9\") pod \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\" (UID: \"7d23ded3-822d-4fad-b59d-f0df6ec4beb4\") "
Sep 09 06:59:55.300553 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.300525    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-config-file" (OuterVolumeSpecName: "config-file") pod "7d23ded3-822d-4fad-b59d-f0df6ec4beb4" (UID: "7d23ded3-822d-4fad-b59d-f0df6ec4beb4"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 06:59:55.302280 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.302254    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-kube-api-access-55rx9" (OuterVolumeSpecName: "kube-api-access-55rx9") pod "7d23ded3-822d-4fad-b59d-f0df6ec4beb4" (UID: "7d23ded3-822d-4fad-b59d-f0df6ec4beb4"). InnerVolumeSpecName "kube-api-access-55rx9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 06:59:55.401428 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.401397    2563 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-config-file\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:59:55.401428 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.401423    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55rx9\" (UniqueName: \"kubernetes.io/projected/7d23ded3-822d-4fad-b59d-f0df6ec4beb4-kube-api-access-55rx9\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 06:59:55.527294 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.527265    2563 generic.go:356] "Generic (PLEG): container finished" podID="7d23ded3-822d-4fad-b59d-f0df6ec4beb4" containerID="3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab" exitCode=0
Sep 09 06:59:55.527676 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.527306    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx"
Sep 09 06:59:55.527676 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.527347    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx" event={"ID":"7d23ded3-822d-4fad-b59d-f0df6ec4beb4","Type":"ContainerDied","Data":"3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab"}
Sep 09 06:59:55.527676 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.527402    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-jsfxx" event={"ID":"7d23ded3-822d-4fad-b59d-f0df6ec4beb4","Type":"ContainerDied","Data":"2eb23d4a2cd4c1babb8db1321ba29d8d876c69714dd39bf1448ad636dd91e4e3"}
Sep 09 06:59:55.527676 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.527426    2563 scope.go:117] "RemoveContainer" containerID="3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab"
Sep 09 06:59:55.536852 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.536838    2563 scope.go:117] "RemoveContainer" containerID="3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab"
Sep 09 06:59:55.537119 ip-10-0-128-206 kubenswrapper[2563]: E0909 06:59:55.537103    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab\": container with ID starting with 3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab not found: ID does not exist" containerID="3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab"
Sep 09 06:59:55.537171 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.537128    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab"} err="failed to get container status \"3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab\": rpc error: code = NotFound desc = could not find container \"3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab\": container with ID starting with 3a912432f420afccca1189d39ec069731ef52c036fa654dd7ad353e577e957ab not found: ID does not exist"
Sep 09 06:59:55.548102 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.548079    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jsfxx"]
Sep 09 06:59:55.552075 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.552030    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-jsfxx"]
Sep 09 06:59:55.973098 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:55.973070    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:56.075065 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.075035    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:56.117183 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.117156    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d23ded3-822d-4fad-b59d-f0df6ec4beb4" path="/var/lib/kubelet/pods/7d23ded3-822d-4fad-b59d-f0df6ec4beb4/volumes"
Sep 09 06:59:56.413885 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.413810    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"]
Sep 09 06:59:56.414296 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.414271    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d23ded3-822d-4fad-b59d-f0df6ec4beb4" containerName="limitador"
Sep 09 06:59:56.414296 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.414294    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d23ded3-822d-4fad-b59d-f0df6ec4beb4" containerName="limitador"
Sep 09 06:59:56.414447 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.414355    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d23ded3-822d-4fad-b59d-f0df6ec4beb4" containerName="limitador"
Sep 09 06:59:56.418818 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.418799    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 06:59:56.421446 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.421425    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 09 06:59:56.421573 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.421537    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-8bmlw\""
Sep 09 06:59:56.421573 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.421558    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 09 06:59:56.421729 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.421555    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 09 06:59:56.425038 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.425016    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"]
Sep 09 06:59:56.446408 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.446382    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 06:59:56.512892 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.512861    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gcmsw\" (UniqueName: \"kubernetes.io/projected/c0ab5869-e46a-4e80-a924-982deffdc0a1-kube-api-access-gcmsw\") pod \"postgres-69fb4594cf-wjtw5\" (UID: \"c0ab5869-e46a-4e80-a924-982deffdc0a1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 06:59:56.513064 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.512926    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c0ab5869-e46a-4e80-a924-982deffdc0a1-data\") pod \"postgres-69fb4594cf-wjtw5\" (UID: \"c0ab5869-e46a-4e80-a924-982deffdc0a1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 06:59:56.613922 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.613886    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gcmsw\" (UniqueName: \"kubernetes.io/projected/c0ab5869-e46a-4e80-a924-982deffdc0a1-kube-api-access-gcmsw\") pod \"postgres-69fb4594cf-wjtw5\" (UID: \"c0ab5869-e46a-4e80-a924-982deffdc0a1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 06:59:56.614278 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.613939    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c0ab5869-e46a-4e80-a924-982deffdc0a1-data\") pod \"postgres-69fb4594cf-wjtw5\" (UID: \"c0ab5869-e46a-4e80-a924-982deffdc0a1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 06:59:56.614319 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.614281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/c0ab5869-e46a-4e80-a924-982deffdc0a1-data\") pod \"postgres-69fb4594cf-wjtw5\" (UID: \"c0ab5869-e46a-4e80-a924-982deffdc0a1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 06:59:56.622454 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.622423    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gcmsw\" (UniqueName: \"kubernetes.io/projected/c0ab5869-e46a-4e80-a924-982deffdc0a1-kube-api-access-gcmsw\") pod \"postgres-69fb4594cf-wjtw5\" (UID: \"c0ab5869-e46a-4e80-a924-982deffdc0a1\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 06:59:56.730298 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.730266    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 06:59:56.999830 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:56.999807    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"]
Sep 09 06:59:57.002159 ip-10-0-128-206 kubenswrapper[2563]: W0909 06:59:57.002126    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0ab5869_e46a_4e80_a924_982deffdc0a1.slice/crio-b15e549cd79682f0e50b8a8322adef45d9a5185c06a5264bb05b39a11eb8f652 WatchSource:0}: Error finding container b15e549cd79682f0e50b8a8322adef45d9a5185c06a5264bb05b39a11eb8f652: Status 404 returned error can't find the container with id b15e549cd79682f0e50b8a8322adef45d9a5185c06a5264bb05b39a11eb8f652
Sep 09 06:59:57.538802 ip-10-0-128-206 kubenswrapper[2563]: I0909 06:59:57.538768    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5" event={"ID":"c0ab5869-e46a-4e80-a924-982deffdc0a1","Type":"ContainerStarted","Data":"b15e549cd79682f0e50b8a8322adef45d9a5185c06a5264bb05b39a11eb8f652"}
Sep 09 07:00:02.568063 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:02.568025    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5" event={"ID":"c0ab5869-e46a-4e80-a924-982deffdc0a1","Type":"ContainerStarted","Data":"700793bcb93a303de0b844d36865598ca1665724314710a44a001f041c6ea5c6"}
Sep 09 07:00:02.568480 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:02.568095    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 07:00:02.585267 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:02.585216    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5" podStartSLOduration=1.855427865 podStartE2EDuration="6.585200019s" totalImagesPullingTime="4.729772154s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 06:59:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:00:02.584226031 +0000 UTC m=+534.954485120" watchObservedRunningTime="2026-09-09 07:00:02.585200019 +0000 UTC m=+534.955459107"
Sep 09 07:00:08.601056 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:08.601025    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-wjtw5"
Sep 09 07:00:08.917017 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:08.916917    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:09.302602 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:09.302528    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:16.822076 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.822042    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:16.854404 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.854375    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-57db658748-vbbj9"]
Sep 09 07:00:16.877944 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.877915    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57db658748-vbbj9"]
Sep 09 07:00:16.878078 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.878031    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:16.880846 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.880821    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dmhkc\""
Sep 09 07:00:16.881031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.880920    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 07:00:16.881499 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.881481    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 07:00:16.881740 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.881708    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 07:00:16.900503 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.900477    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-webhook-cert\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:16.900623 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.900609    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-metrics-certs\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:16.900696 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.900651    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lclrr\" (UniqueName: \"kubernetes.io/projected/1930f63e-f0d0-442c-850a-065108c89bcd-kube-api-access-lclrr\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:16.903496 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.903474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:16.964481 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:16.964449    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:17.001348 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.001310    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-metrics-certs\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:17.001348 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.001352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lclrr\" (UniqueName: \"kubernetes.io/projected/1930f63e-f0d0-442c-850a-065108c89bcd-kube-api-access-lclrr\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:17.001642 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.001389    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-webhook-cert\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:17.003983 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.003961    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-metrics-certs\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:17.004084 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.004032    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-webhook-cert\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:17.011095 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.011064    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lclrr\" (UniqueName: \"kubernetes.io/projected/1930f63e-f0d0-442c-850a-065108c89bcd-kube-api-access-lclrr\") pod \"maas-controller-57db658748-vbbj9\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") " pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:17.027445 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.027414    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8cc576b77-mmjqb"]
Sep 09 07:00:17.031189 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.031162    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.043001 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.042980    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8cc576b77-mmjqb"]
Sep 09 07:00:17.045292 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.045270    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:17.102075 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.102011    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-metrics-certs\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.102075 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.102038    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-webhook-cert\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.102075 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.102067    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpz2n\" (UniqueName: \"kubernetes.io/projected/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-kube-api-access-tpz2n\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.189061 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.189015    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:17.203453 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.203427    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-metrics-certs\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.203560 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.203460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-webhook-cert\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.203560 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.203498    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpz2n\" (UniqueName: \"kubernetes.io/projected/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-kube-api-access-tpz2n\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.206154 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.206131    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-metrics-certs\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.206395 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.206375    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-webhook-cert\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.213442 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.213420    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpz2n\" (UniqueName: \"kubernetes.io/projected/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-kube-api-access-tpz2n\") pod \"maas-controller-8cc576b77-mmjqb\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") " pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.341917 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.341875    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:17.462130 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.462105    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-57db658748-vbbj9"]
Sep 09 07:00:17.464541 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:00:17.464513    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1930f63e_f0d0_442c_850a_065108c89bcd.slice/crio-78393c6dfbbf0d4a0151063d8e398172d22cd5a891c0c07c57960d9d1c03588a WatchSource:0}: Error finding container 78393c6dfbbf0d4a0151063d8e398172d22cd5a891c0c07c57960d9d1c03588a: Status 404 returned error can't find the container with id 78393c6dfbbf0d4a0151063d8e398172d22cd5a891c0c07c57960d9d1c03588a
Sep 09 07:00:17.611819 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.611794    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8cc576b77-mmjqb"]
Sep 09 07:00:17.613663 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:00:17.613638    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode008d526_69c5_4d6f_ad4d_d4b404d40ff8.slice/crio-ddc70132753bf527403b473c55fe545518754c868dc8b47565746fa9676fcde4 WatchSource:0}: Error finding container ddc70132753bf527403b473c55fe545518754c868dc8b47565746fa9676fcde4: Status 404 returned error can't find the container with id ddc70132753bf527403b473c55fe545518754c868dc8b47565746fa9676fcde4
Sep 09 07:00:17.638184 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.638156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57db658748-vbbj9" event={"ID":"1930f63e-f0d0-442c-850a-065108c89bcd","Type":"ContainerStarted","Data":"78393c6dfbbf0d4a0151063d8e398172d22cd5a891c0c07c57960d9d1c03588a"}
Sep 09 07:00:17.639256 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:17.639234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-mmjqb" event={"ID":"e008d526-69c5-4d6f-ad4d-d4b404d40ff8","Type":"ContainerStarted","Data":"ddc70132753bf527403b473c55fe545518754c868dc8b47565746fa9676fcde4"}
Sep 09 07:00:18.601341 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:18.601303    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:18.703894 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:18.703855    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:20.565985 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:20.565954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:20.665285 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:20.665186    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57db658748-vbbj9" event={"ID":"1930f63e-f0d0-442c-850a-065108c89bcd","Type":"ContainerStarted","Data":"502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c"}
Sep 09 07:00:20.665285 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:20.665277    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:20.666832 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:20.666799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-mmjqb" event={"ID":"e008d526-69c5-4d6f-ad4d-d4b404d40ff8","Type":"ContainerStarted","Data":"34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797"}
Sep 09 07:00:20.666957 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:20.666839    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:20.666957 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:20.666864    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:20.686492 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:20.686442    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-57db658748-vbbj9" podStartSLOduration=1.7627639529999999 podStartE2EDuration="4.686425527s" totalImagesPullingTime="2.923661574s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:00:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:00:20.684646426 +0000 UTC m=+553.054905513" watchObservedRunningTime="2026-09-09 07:00:20.686425527 +0000 UTC m=+553.056684615"
Sep 09 07:00:20.707806 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:20.707750    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8cc576b77-mmjqb" podStartSLOduration=0.926326774 podStartE2EDuration="3.70772831s" totalImagesPullingTime="2.781401536s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:00:20.707034374 +0000 UTC m=+553.077293677" watchObservedRunningTime="2026-09-09 07:00:20.70772831 +0000 UTC m=+553.077987399"
Sep 09 07:00:31.673395 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:31.673362    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:31.673961 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:31.673444    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:00:31.738737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:31.738700    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57db658748-vbbj9"]
Sep 09 07:00:31.739198 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:31.739142    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-57db658748-vbbj9" podUID="1930f63e-f0d0-442c-850a-065108c89bcd" containerName="manager" containerID="cri-o://502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c" gracePeriod=30
Sep 09 07:00:31.986986 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:31.986959    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:32.041779 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.041743    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lclrr\" (UniqueName: \"kubernetes.io/projected/1930f63e-f0d0-442c-850a-065108c89bcd-kube-api-access-lclrr\") pod \"1930f63e-f0d0-442c-850a-065108c89bcd\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") "
Sep 09 07:00:32.041966 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.041834    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-webhook-cert\") pod \"1930f63e-f0d0-442c-850a-065108c89bcd\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") "
Sep 09 07:00:32.041966 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.041853    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-metrics-certs\") pod \"1930f63e-f0d0-442c-850a-065108c89bcd\" (UID: \"1930f63e-f0d0-442c-850a-065108c89bcd\") "
Sep 09 07:00:32.044065 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.044032    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1930f63e-f0d0-442c-850a-065108c89bcd" (UID: "1930f63e-f0d0-442c-850a-065108c89bcd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:00:32.044189 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.044100    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1930f63e-f0d0-442c-850a-065108c89bcd-kube-api-access-lclrr" (OuterVolumeSpecName: "kube-api-access-lclrr") pod "1930f63e-f0d0-442c-850a-065108c89bcd" (UID: "1930f63e-f0d0-442c-850a-065108c89bcd"). InnerVolumeSpecName "kube-api-access-lclrr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:00:32.044189 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.044131    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1930f63e-f0d0-442c-850a-065108c89bcd" (UID: "1930f63e-f0d0-442c-850a-065108c89bcd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:00:32.142741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.142714    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lclrr\" (UniqueName: \"kubernetes.io/projected/1930f63e-f0d0-442c-850a-065108c89bcd-kube-api-access-lclrr\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:00:32.142741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.142738    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-webhook-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:00:32.142883 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.142752    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1930f63e-f0d0-442c-850a-065108c89bcd-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:00:32.729184 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.729147    2563 generic.go:356] "Generic (PLEG): container finished" podID="1930f63e-f0d0-442c-850a-065108c89bcd" containerID="502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c" exitCode=0
Sep 09 07:00:32.729682 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.729191    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-57db658748-vbbj9"
Sep 09 07:00:32.729682 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.729230    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57db658748-vbbj9" event={"ID":"1930f63e-f0d0-442c-850a-065108c89bcd","Type":"ContainerDied","Data":"502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c"}
Sep 09 07:00:32.729682 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.729291    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-57db658748-vbbj9" event={"ID":"1930f63e-f0d0-442c-850a-065108c89bcd","Type":"ContainerDied","Data":"78393c6dfbbf0d4a0151063d8e398172d22cd5a891c0c07c57960d9d1c03588a"}
Sep 09 07:00:32.729682 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.729324    2563 scope.go:117] "RemoveContainer" containerID="502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c"
Sep 09 07:00:32.737857 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.737842    2563 scope.go:117] "RemoveContainer" containerID="502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c"
Sep 09 07:00:32.738136 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:00:32.738116    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c\": container with ID starting with 502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c not found: ID does not exist" containerID="502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c"
Sep 09 07:00:32.738213 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.738143    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c"} err="failed to get container status \"502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c\": rpc error: code = NotFound desc = could not find container \"502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c\": container with ID starting with 502d7e80c06c404c7a4a6c00e28c078c6ad893f6e58dc77e454d7ae7dd2b523c not found: ID does not exist"
Sep 09 07:00:32.745555 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.745335    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-57db658748-vbbj9"]
Sep 09 07:00:32.747470 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:32.747450    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-57db658748-vbbj9"]
Sep 09 07:00:34.116944 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:34.116901    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1930f63e-f0d0-442c-850a-065108c89bcd" path="/var/lib/kubelet/pods/1930f63e-f0d0-442c-850a-065108c89bcd/volumes"
Sep 09 07:00:52.156838 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:52.156796    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:52.277554 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:52.277506    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:53.623697 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.623651    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:53.725044 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.725010    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:53.825215 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.825185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:53.861321 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.861292    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-7998845878-sbdx4"]
Sep 09 07:00:53.861698 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.861685    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1930f63e-f0d0-442c-850a-065108c89bcd" containerName="manager"
Sep 09 07:00:53.861754 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.861700    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="1930f63e-f0d0-442c-850a-065108c89bcd" containerName="manager"
Sep 09 07:00:53.861789 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.861767    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="1930f63e-f0d0-442c-850a-065108c89bcd" containerName="manager"
Sep 09 07:00:53.867796 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.867777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:53.871382 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.871362    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 09 07:00:53.871519 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.871405    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-lbttc\""
Sep 09 07:00:53.871519 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.871462    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 09 07:00:53.875026 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.874971    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7998845878-sbdx4"]
Sep 09 07:00:53.902811 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:53.902788    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:54.036815 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.036780    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff9zw\" (UniqueName: \"kubernetes.io/projected/c790ded1-63e5-4620-af5f-8274553be84a-kube-api-access-ff9zw\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.036815 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.036819    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-metrics-certs\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.037005 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.036906    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c790ded1-63e5-4620-af5f-8274553be84a-openshift-service-ca\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.037005 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.036955    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-maas-api-tls\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.137434 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.137352    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c790ded1-63e5-4620-af5f-8274553be84a-openshift-service-ca\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.137434 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.137410    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-maas-api-tls\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.137694 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.137452    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff9zw\" (UniqueName: \"kubernetes.io/projected/c790ded1-63e5-4620-af5f-8274553be84a-kube-api-access-ff9zw\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.137694 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.137484    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-metrics-certs\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.138089 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.138067    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c790ded1-63e5-4620-af5f-8274553be84a-openshift-service-ca\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.140050 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.140026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-maas-api-tls\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.140161 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.140057    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-metrics-certs\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.145066 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.145041    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff9zw\" (UniqueName: \"kubernetes.io/projected/c790ded1-63e5-4620-af5f-8274553be84a-kube-api-access-ff9zw\") pod \"maas-api-7998845878-sbdx4\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") " pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.179493 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.179467    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:54.231058 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.230920    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:54.330127 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.330089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:00:54.419772 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.419694    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"]
Sep 09 07:00:54.423404 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.423388    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:54.426237 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.426207    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-fcqs8\""
Sep 09 07:00:54.426348 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.426337    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 09 07:00:54.433561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.433536    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"]
Sep 09 07:00:54.460730 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.460709    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-7998845878-sbdx4"]
Sep 09 07:00:54.464183 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:00:54.464148    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc790ded1_63e5_4620_af5f_8274553be84a.slice/crio-9d244e4d6bb50d0ee7f06ff66e4645574d22507baab2ff3a2932e56eadce6903 WatchSource:0}: Error finding container 9d244e4d6bb50d0ee7f06ff66e4645574d22507baab2ff3a2932e56eadce6903: Status 404 returned error can't find the container with id 9d244e4d6bb50d0ee7f06ff66e4645574d22507baab2ff3a2932e56eadce6903
Sep 09 07:00:54.540646 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.540613    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/328d013c-a98f-46aa-99ae-7edd3dfc8e9e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-66x5x\" (UID: \"328d013c-a98f-46aa-99ae-7edd3dfc8e9e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:54.540812 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.540776    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7rbjk\" (UniqueName: \"kubernetes.io/projected/328d013c-a98f-46aa-99ae-7edd3dfc8e9e-kube-api-access-7rbjk\") pod \"payload-pre-processing-59fbc47bf8-66x5x\" (UID: \"328d013c-a98f-46aa-99ae-7edd3dfc8e9e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:54.641714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.641680    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7rbjk\" (UniqueName: \"kubernetes.io/projected/328d013c-a98f-46aa-99ae-7edd3dfc8e9e-kube-api-access-7rbjk\") pod \"payload-pre-processing-59fbc47bf8-66x5x\" (UID: \"328d013c-a98f-46aa-99ae-7edd3dfc8e9e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:54.642065 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.641722    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/328d013c-a98f-46aa-99ae-7edd3dfc8e9e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-66x5x\" (UID: \"328d013c-a98f-46aa-99ae-7edd3dfc8e9e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:54.642376 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.642356    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/328d013c-a98f-46aa-99ae-7edd3dfc8e9e-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-66x5x\" (UID: \"328d013c-a98f-46aa-99ae-7edd3dfc8e9e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:54.655731 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.655703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7rbjk\" (UniqueName: \"kubernetes.io/projected/328d013c-a98f-46aa-99ae-7edd3dfc8e9e-kube-api-access-7rbjk\") pod \"payload-pre-processing-59fbc47bf8-66x5x\" (UID: \"328d013c-a98f-46aa-99ae-7edd3dfc8e9e\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:54.736292 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.736258    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:54.840739 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:54.840708    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4" event={"ID":"c790ded1-63e5-4620-af5f-8274553be84a","Type":"ContainerStarted","Data":"9d244e4d6bb50d0ee7f06ff66e4645574d22507baab2ff3a2932e56eadce6903"}
Sep 09 07:00:55.007761 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:55.007693    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"]
Sep 09 07:00:55.010032 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:00:55.010002    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328d013c_a98f_46aa_99ae_7edd3dfc8e9e.slice/crio-8aed0459d795a8165f275fe1517e1d5d151f395064bdcee2d62d136611d772c6 WatchSource:0}: Error finding container 8aed0459d795a8165f275fe1517e1d5d151f395064bdcee2d62d136611d772c6: Status 404 returned error can't find the container with id 8aed0459d795a8165f275fe1517e1d5d151f395064bdcee2d62d136611d772c6
Sep 09 07:00:55.849981 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:55.849855    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x" event={"ID":"328d013c-a98f-46aa-99ae-7edd3dfc8e9e","Type":"ContainerStarted","Data":"8aed0459d795a8165f275fe1517e1d5d151f395064bdcee2d62d136611d772c6"}
Sep 09 07:00:56.857605 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:56.857551    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x" event={"ID":"328d013c-a98f-46aa-99ae-7edd3dfc8e9e","Type":"ContainerStarted","Data":"245711395fa067e5b7673f314177a5ccf735f4923311e24e7579a209db3cf6da"}
Sep 09 07:00:56.858184 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:56.857709    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:00:56.859577 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:56.859546    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4" event={"ID":"c790ded1-63e5-4620-af5f-8274553be84a","Type":"ContainerStarted","Data":"43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7"}
Sep 09 07:00:56.859723 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:56.859658    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:00:56.876775 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:56.876690    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x" podStartSLOduration=1.217733728 podStartE2EDuration="2.876677027s" totalImagesPullingTime="1.658943299s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:00:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:00:56.875970139 +0000 UTC m=+589.246229230" watchObservedRunningTime="2026-09-09 07:00:56.876677027 +0000 UTC m=+589.246936114"
Sep 09 07:00:56.893194 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:00:56.893148    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4" podStartSLOduration=1.73999829 podStartE2EDuration="3.893132452s" totalImagesPullingTime="2.153134162s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:00:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:00:56.892050947 +0000 UTC m=+589.262310035" watchObservedRunningTime="2026-09-09 07:00:56.893132452 +0000 UTC m=+589.263391552"
Sep 09 07:01:02.870412 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:02.870378    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:01:03.988361 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:03.988301    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:01:04.084768 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:04.084723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:01:07.867351 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:07.867320    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-66x5x"
Sep 09 07:01:08.098402 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:08.098373    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:01:08.101402 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:08.101383    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:01:23.232126 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.232091    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"]
Sep 09 07:01:23.235513 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.235496    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.239247 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.239218    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 09 07:01:23.239397 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.239317    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 09 07:01:23.239397 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.239351    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 09 07:01:23.239529 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.239427    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nnc67\""
Sep 09 07:01:23.245081 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.245057    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"]
Sep 09 07:01:23.301222 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.301190    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.301222 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.301228    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c1f79d-5cbf-488d-b0bc-84cd525976fa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.301432 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.301264    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.301432 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.301280    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.301432 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.301369    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-722j7\" (UniqueName: \"kubernetes.io/projected/03c1f79d-5cbf-488d-b0bc-84cd525976fa-kube-api-access-722j7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.301559 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.301473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.331337 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.331306    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:01:23.402176 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402136    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.402176 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402171    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c1f79d-5cbf-488d-b0bc-84cd525976fa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.402411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402231    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.402411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.402411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402291    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-722j7\" (UniqueName: \"kubernetes.io/projected/03c1f79d-5cbf-488d-b0bc-84cd525976fa-kube-api-access-722j7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.402411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402344    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.402784 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402763    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.402887 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.402887 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.402828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.404493 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.404464    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/03c1f79d-5cbf-488d-b0bc-84cd525976fa-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.404670 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.404650    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/03c1f79d-5cbf-488d-b0bc-84cd525976fa-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.413306 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.413281    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-722j7\" (UniqueName: \"kubernetes.io/projected/03c1f79d-5cbf-488d-b0bc-84cd525976fa-kube-api-access-722j7\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb\" (UID: \"03c1f79d-5cbf-488d-b0bc-84cd525976fa\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.546843 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.546739    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:23.823147 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.823080    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"]
Sep 09 07:01:23.825876 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:01:23.825844    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c1f79d_5cbf_488d_b0bc_84cd525976fa.slice/crio-be08bc48c1ec08e29dbe91621d783addb9b52c8d6751d64372288e3d84389bb7 WatchSource:0}: Error finding container be08bc48c1ec08e29dbe91621d783addb9b52c8d6751d64372288e3d84389bb7: Status 404 returned error can't find the container with id be08bc48c1ec08e29dbe91621d783addb9b52c8d6751d64372288e3d84389bb7
Sep 09 07:01:23.993193 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:23.993159    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb" event={"ID":"03c1f79d-5cbf-488d-b0bc-84cd525976fa","Type":"ContainerStarted","Data":"be08bc48c1ec08e29dbe91621d783addb9b52c8d6751d64372288e3d84389bb7"}
Sep 09 07:01:25.000067 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:25.000034    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb" event={"ID":"03c1f79d-5cbf-488d-b0bc-84cd525976fa","Type":"ContainerStarted","Data":"dc4c792d3e165591c6ebb4760db88bb21ced6b0a4ac10abc6c7f1e93969f299e"}
Sep 09 07:01:25.017834 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:25.017750    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb" podStartSLOduration=0.960417175 podStartE2EDuration="2.017729843s" totalImagesPullingTime="1.057312668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:01:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:01:25.016836259 +0000 UTC m=+617.387095348" watchObservedRunningTime="2026-09-09 07:01:25.017729843 +0000 UTC m=+617.387988931"
Sep 09 07:01:29.625001 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.624969    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"]
Sep 09 07:01:29.631486 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.631460    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.634029 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.634004    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 09 07:01:29.640238 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.640214    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"]
Sep 09 07:01:29.663233 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.663206    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:01:29.763428 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.763395    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.763603 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.763449    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.763603 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.763484    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f27jd\" (UniqueName: \"kubernetes.io/projected/3c278d9b-2f37-4920-adfc-74ffb69a3e10-kube-api-access-f27jd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.763603 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.763560    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.763727 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.763621    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c278d9b-2f37-4920-adfc-74ffb69a3e10-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.763727 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.763677    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.864936 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.864900    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.865103 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.864966    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.865103 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.864990    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f27jd\" (UniqueName: \"kubernetes.io/projected/3c278d9b-2f37-4920-adfc-74ffb69a3e10-kube-api-access-f27jd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.865225 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.865206    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.865341 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.865262    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c278d9b-2f37-4920-adfc-74ffb69a3e10-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.865341 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.865304    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.865341 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.865316    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.865471 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.865342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.865521 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.865504    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.867498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.867467    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c278d9b-2f37-4920-adfc-74ffb69a3e10-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.867751 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.867736    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c278d9b-2f37-4920-adfc-74ffb69a3e10-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.873118 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.873096    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f27jd\" (UniqueName: \"kubernetes.io/projected/3c278d9b-2f37-4920-adfc-74ffb69a3e10-kube-api-access-f27jd\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5bvqd\" (UID: \"3c278d9b-2f37-4920-adfc-74ffb69a3e10\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:29.943755 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:29.943735    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:30.228102 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:30.227989    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"]
Sep 09 07:01:30.230521 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:01:30.230491    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c278d9b_2f37_4920_adfc_74ffb69a3e10.slice/crio-efebec3ef74bf1c70c2cb6db55a18bcf5e94c6c25a8519544b2ba353ba951296 WatchSource:0}: Error finding container efebec3ef74bf1c70c2cb6db55a18bcf5e94c6c25a8519544b2ba353ba951296: Status 404 returned error can't find the container with id efebec3ef74bf1c70c2cb6db55a18bcf5e94c6c25a8519544b2ba353ba951296
Sep 09 07:01:31.032100 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:31.032059    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd" event={"ID":"3c278d9b-2f37-4920-adfc-74ffb69a3e10","Type":"ContainerStarted","Data":"71211f03b57f2c77245cc67a46e623a30128d965af29db21b4214492c2578008"}
Sep 09 07:01:31.032481 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:31.032125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd" event={"ID":"3c278d9b-2f37-4920-adfc-74ffb69a3e10","Type":"ContainerStarted","Data":"efebec3ef74bf1c70c2cb6db55a18bcf5e94c6c25a8519544b2ba353ba951296"}
Sep 09 07:01:31.050269 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:31.050208    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd" podStartSLOduration=1.8738034780000001 podStartE2EDuration="2.050191288s" totalImagesPullingTime="176.38781ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:01:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:01:31.04861046 +0000 UTC m=+623.418869549" watchObservedRunningTime="2026-09-09 07:01:31.050191288 +0000 UTC m=+623.420450376"
Sep 09 07:01:33.547204 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:33.547157    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:33.547642 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:33.547236    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:33.559733 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:33.559710    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:34.058732 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:34.058704    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb"
Sep 09 07:01:35.446927 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:35.446880    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:01:39.944340 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:39.944296    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:39.944871 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:39.944376    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:39.956978 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:39.956952    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:40.094067 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:40.094035    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5bvqd"
Sep 09 07:01:42.278885 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:42.278840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:01:52.444207 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:52.444173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:01:54.306390 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:01:54.306352    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:02:01.326212 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.326172    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"]
Sep 09 07:02:01.332202 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.332180    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.334652 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.334626    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 09 07:02:01.341248 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.341225    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"]
Sep 09 07:02:01.358282 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.358253    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:02:01.463879 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.463835    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vwgt\" (UniqueName: \"kubernetes.io/projected/d42244bc-2de3-435a-8037-a5d77aa39b3f-kube-api-access-7vwgt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.463879 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.463882    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d42244bc-2de3-435a-8037-a5d77aa39b3f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.464141 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.464022    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.464141 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.464058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.464141 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.464082    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.464287 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.464155    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.565290 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565248    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.565290 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565297    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vwgt\" (UniqueName: \"kubernetes.io/projected/d42244bc-2de3-435a-8037-a5d77aa39b3f-kube-api-access-7vwgt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.565530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565322    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d42244bc-2de3-435a-8037-a5d77aa39b3f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.565530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565384    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.565530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565400    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.565530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565422    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.565871 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565843    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.566003 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.566003 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.565948    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.567674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.567651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d42244bc-2de3-435a-8037-a5d77aa39b3f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.567961 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.567942    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d42244bc-2de3-435a-8037-a5d77aa39b3f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.573453 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.573430    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vwgt\" (UniqueName: \"kubernetes.io/projected/d42244bc-2de3-435a-8037-a5d77aa39b3f-kube-api-access-7vwgt\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5s44q\" (UID: \"d42244bc-2de3-435a-8037-a5d77aa39b3f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.645331 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.645228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:01.919393 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.919366    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"]
Sep 09 07:02:01.922299 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:02:01.922269    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd42244bc_2de3_435a_8037_a5d77aa39b3f.slice/crio-d91b66903fe769a6f90171ab08c863d22a15dbbdb58974b22a8cdf930ac2aa3d WatchSource:0}: Error finding container d91b66903fe769a6f90171ab08c863d22a15dbbdb58974b22a8cdf930ac2aa3d: Status 404 returned error can't find the container with id d91b66903fe769a6f90171ab08c863d22a15dbbdb58974b22a8cdf930ac2aa3d
Sep 09 07:02:01.923813 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:01.923792    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 07:02:02.200805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:02.200771    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q" event={"ID":"d42244bc-2de3-435a-8037-a5d77aa39b3f","Type":"ContainerStarted","Data":"0c8400a08ec5a6fe99e16764372d23234423b5d0918443369e3df97eaf534623"}
Sep 09 07:02:02.200954 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:02.200826    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q" event={"ID":"d42244bc-2de3-435a-8037-a5d77aa39b3f","Type":"ContainerStarted","Data":"d91b66903fe769a6f90171ab08c863d22a15dbbdb58974b22a8cdf930ac2aa3d"}
Sep 09 07:02:02.221077 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:02.221007    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q" podStartSLOduration=1.020003915 podStartE2EDuration="1.220990119s" totalImagesPullingTime="200.986204ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:02:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:02:02.219835441 +0000 UTC m=+654.590094529" watchObservedRunningTime="2026-09-09 07:02:02.220990119 +0000 UTC m=+654.591249206"
Sep 09 07:02:11.646270 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:11.646230    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:11.646832 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:11.646336    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:11.658557 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:11.658525    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:12.260787 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:12.260753    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5s44q"
Sep 09 07:02:24.458674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:24.458639    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:02:36.405711 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:02:36.405673    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:03:06.560410 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:03:06.560330    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:03:21.865400 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:03:21.865365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:03:43.843915 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:03:43.843879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:03:43.953339 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:03:43.953305    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:03:44.584147 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:03:44.584118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:03:44.676059 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:03:44.676026    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:03:45.758031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:03:45.758001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:03:45.845036 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:03:45.845003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:07.163264 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:07.163228    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:07.288448 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:07.288420    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:07.665335 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:07.665302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:07.908762 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:07.908727    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:08.733982 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:08.733947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:08.834830 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:08.834797    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:09.285828 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:09.285798    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:10.984550 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:10.984516    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:11.068622 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:11.068574    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:12.211679 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.211642    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"]
Sep 09 07:04:12.215987 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.215965    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.218369 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.218345    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-b212cb-serving-cert\""
Sep 09 07:04:12.231508 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.231481    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"]
Sep 09 07:04:12.247305 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.247273    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:12.304659 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.304623    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6xx26\" (UniqueName: \"kubernetes.io/projected/f0033853-68db-42f6-8da6-41b9d1d642c3-kube-api-access-6xx26\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.304803 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.304767    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.304862 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.304833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0033853-68db-42f6-8da6-41b9d1d642c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.304909 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.304892    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:12.304947 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.304909    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-metrics-certs\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.405454 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.405414    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6xx26\" (UniqueName: \"kubernetes.io/projected/f0033853-68db-42f6-8da6-41b9d1d642c3-kube-api-access-6xx26\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.405680 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.405525    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.405680 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.405571    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0033853-68db-42f6-8da6-41b9d1d642c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.405680 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.405643    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-metrics-certs\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.405907 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.405885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:12.406347 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.406317    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0033853-68db-42f6-8da6-41b9d1d642c3-openshift-service-ca\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.408092 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.408071    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-metrics-certs\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.408186 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.408155    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-maas-api-tls\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.413870 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.413846    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6xx26\" (UniqueName: \"kubernetes.io/projected/f0033853-68db-42f6-8da6-41b9d1d642c3-kube-api-access-6xx26\") pod \"maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.526654 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.526566    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:12.729390 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.729365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:12.802979 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.802946    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"]
Sep 09 07:04:12.804284 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.804248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:12.804706 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:04:12.804682    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0033853_68db_42f6_8da6_41b9d1d642c3.slice/crio-cd59d59f46a43db6a8809cb08c00c7030e10f27bb06ba2963ce23dc6b759e79f WatchSource:0}: Error finding container cd59d59f46a43db6a8809cb08c00c7030e10f27bb06ba2963ce23dc6b759e79f: Status 404 returned error can't find the container with id cd59d59f46a43db6a8809cb08c00c7030e10f27bb06ba2963ce23dc6b759e79f
Sep 09 07:04:12.842689 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:12.842656    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f" event={"ID":"f0033853-68db-42f6-8da6-41b9d1d642c3","Type":"ContainerStarted","Data":"cd59d59f46a43db6a8809cb08c00c7030e10f27bb06ba2963ce23dc6b759e79f"}
Sep 09 07:04:15.074405 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:15.074373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:15.175829 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:15.175782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:15.313239 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:15.313154    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:15.693703 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:15.693669    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:15.770428 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:15.770399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:15.858271 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:15.858234    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f" event={"ID":"f0033853-68db-42f6-8da6-41b9d1d642c3","Type":"ContainerStarted","Data":"1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3"}
Sep 09 07:04:15.858457 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:15.858351    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:15.876234 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:15.876191    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f" podStartSLOduration=1.6265398439999998 podStartE2EDuration="3.876177876s" totalImagesPullingTime="2.249638032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:04:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:04:15.875643982 +0000 UTC m=+788.245903072" watchObservedRunningTime="2026-09-09 07:04:15.876177876 +0000 UTC m=+788.246436963"
Sep 09 07:04:18.342629 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:18.342578    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:18.441179 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:18.441143    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:19.608482 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.608447    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"]
Sep 09 07:04:19.612428 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.612405    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.614762 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.614737    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-ef07c382-serving-cert\""
Sep 09 07:04:19.624467 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.624440    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"]
Sep 09 07:04:19.651789 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.651757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:19.673856 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.673827    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-openshift-service-ca\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.673856 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.673857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-metrics-certs\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.674018 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.673892    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vgtqm\" (UniqueName: \"kubernetes.io/projected/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-kube-api-access-vgtqm\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.674018 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.673961    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.752754 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.752723    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:19.774985 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.774959    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-openshift-service-ca\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.775090 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.774992    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-metrics-certs\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.775090 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.775033    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vgtqm\" (UniqueName: \"kubernetes.io/projected/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-kube-api-access-vgtqm\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.775090 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.775057    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.775196 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:04:19.775141    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-serving-cert: secret "maas-api-e2e-ait-ef07c382-serving-cert" not found
Sep 09 07:04:19.775196 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:04:19.775192    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls podName:fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c nodeName:}" failed. No retries permitted until 2026-09-09 07:04:20.275173337 +0000 UTC m=+792.645432407 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls") pod "maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v" (UID: "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c") : secret "maas-api-e2e-ait-ef07c382-serving-cert" not found
Sep 09 07:04:19.775799 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.775769    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-openshift-service-ca\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.777364 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.777343    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-metrics-certs\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.783348 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.783326    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vgtqm\" (UniqueName: \"kubernetes.io/projected/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-kube-api-access-vgtqm\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:19.853764 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.853734    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:19.954736 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:19.954700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:20.077186 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.077131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:20.168995 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.168943    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:20.252757 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.252675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:20.280077 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.280042    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:20.282564 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.282537    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls\") pod \"maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:20.350803 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.350771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:20.525674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.525558    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:20.801333 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.801304    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"]
Sep 09 07:04:20.802936 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:04:20.802904    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa4ebc4c_10e5_4d84_bd1a_1ebdefb8d37c.slice/crio-5201bed8678555baa8c6ee442d0ee7d04261a1c47db1fb9687872d01159c6452 WatchSource:0}: Error finding container 5201bed8678555baa8c6ee442d0ee7d04261a1c47db1fb9687872d01159c6452: Status 404 returned error can't find the container with id 5201bed8678555baa8c6ee442d0ee7d04261a1c47db1fb9687872d01159c6452
Sep 09 07:04:20.883193 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:20.883162    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v" event={"ID":"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c","Type":"ContainerStarted","Data":"5201bed8678555baa8c6ee442d0ee7d04261a1c47db1fb9687872d01159c6452"}
Sep 09 07:04:21.869023 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:21.868993    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:04:21.891811 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:21.891778    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v" event={"ID":"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c","Type":"ContainerStarted","Data":"f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0"}
Sep 09 07:04:21.891959 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:21.891939    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:21.919083 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:21.919020    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v" podStartSLOduration=2.481423053 podStartE2EDuration="2.918998784s" totalImagesPullingTime="437.575731ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:04:21.915567188 +0000 UTC m=+794.285826276" watchObservedRunningTime="2026-09-09 07:04:21.918998784 +0000 UTC m=+794.289257871"
Sep 09 07:04:23.051312 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:23.051266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:23.147323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:23.147280    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:24.134923 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.134891    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:24.181187 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.181155    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"]
Sep 09 07:04:24.185431 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.185405    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.188220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.188199    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-40c98b-serving-cert\""
Sep 09 07:04:24.197113 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.197085    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"]
Sep 09 07:04:24.220698 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.220660    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-openshift-service-ca\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.220839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.220719    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-metrics-certs\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.220839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.220773    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.220914 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.220869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkbgp\" (UniqueName: \"kubernetes.io/projected/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-kube-api-access-hkbgp\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.240372 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.240340    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:24.321655 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.321625    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.321827 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.321701    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkbgp\" (UniqueName: \"kubernetes.io/projected/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-kube-api-access-hkbgp\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.321827 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.321734    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-openshift-service-ca\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.321827 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:04:24.321771    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-serving-cert: secret "maas-api-e2e-autores-40c98b-serving-cert" not found
Sep 09 07:04:24.321989 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:04:24.321830    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls podName:ca70c1d5-01f3-4fd9-b0bf-69c4a425314b nodeName:}" failed. No retries permitted until 2026-09-09 07:04:24.821813635 +0000 UTC m=+797.192072705 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls") pod "maas-api-e2e-autores-40c98b-6bdc659744-28pgn" (UID: "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b") : secret "maas-api-e2e-autores-40c98b-serving-cert" not found
Sep 09 07:04:24.321989 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.321876    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-metrics-certs\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.322364 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.322342    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-openshift-service-ca\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.324432 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.324410    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-metrics-certs\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.330544 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.330522    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkbgp\" (UniqueName: \"kubernetes.io/projected/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-kube-api-access-hkbgp\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.337209 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.337176    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:24.433639 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.433598    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:24.800544 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.800459    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:24.827225 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.827185    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.829842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.829817    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls\") pod \"maas-api-e2e-autores-40c98b-6bdc659744-28pgn\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:24.904146 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:24.904110    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:25.100457 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:25.100377    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:25.387762 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:25.387737    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"]
Sep 09 07:04:25.390004 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:04:25.389977    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca70c1d5_01f3_4fd9_b0bf_69c4a425314b.slice/crio-73ecf6bcf9970dbca70e1f7f140e781bf5912044673e45d9ec0f0d613b339caf WatchSource:0}: Error finding container 73ecf6bcf9970dbca70e1f7f140e781bf5912044673e45d9ec0f0d613b339caf: Status 404 returned error can't find the container with id 73ecf6bcf9970dbca70e1f7f140e781bf5912044673e45d9ec0f0d613b339caf
Sep 09 07:04:25.912693 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:25.912657    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn" event={"ID":"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b","Type":"ContainerStarted","Data":"73ecf6bcf9970dbca70e1f7f140e781bf5912044673e45d9ec0f0d613b339caf"}
Sep 09 07:04:25.931637 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:25.931572    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:26.105873 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:26.105838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:26.202534 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:26.202496    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:26.299287 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:26.299250    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:26.919655 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:26.919554    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn" event={"ID":"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b","Type":"ContainerStarted","Data":"2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035"}
Sep 09 07:04:26.919655 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:26.919647    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:26.946787 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:26.946742    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn" podStartSLOduration=1.9207791950000002 podStartE2EDuration="2.94672672s" totalImagesPullingTime="1.025947525s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:04:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:04:26.944785836 +0000 UTC m=+799.315044945" watchObservedRunningTime="2026-09-09 07:04:26.94672672 +0000 UTC m=+799.316985807"
Sep 09 07:04:27.901898 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:27.901868    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:04:29.045113 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:29.045054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:30.746061 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:30.745959    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:30.928146 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:30.928074    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:31.053212 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:31.053108    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:31.148211 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:31.148171    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:31.247864 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:31.247833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:32.930692 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:32.930659    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:04:34.332084 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:34.332042    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:35.926683 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:35.926642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:36.321298 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:36.321209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:36.605705 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:36.605624    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:36.760523 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:36.760490    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:37.726709 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:37.726678    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:37.826545 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:37.826511    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:40.075138 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:40.075102    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:40.220770 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:40.220726    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:40.666241 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:40.666209    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:41.516790 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.516759    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"]
Sep 09 07:04:41.521031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.521008    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.524174 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.524081    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-fc9900-serving-cert\""
Sep 09 07:04:41.535097 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.535065    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"]
Sep 09 07:04:41.555439 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.555396    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:41.590900 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.590866    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.591057 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.590923    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2r4f4\" (UniqueName: \"kubernetes.io/projected/a5cc1a83-5420-4190-8e72-5eb49f523597-kube-api-access-2r4f4\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.591057 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.590988    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cc1a83-5420-4190-8e72-5eb49f523597-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.591140 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.591062    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-metrics-certs\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.655055 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.655012    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:41.691936 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.691902    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.692123 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.691965    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2r4f4\" (UniqueName: \"kubernetes.io/projected/a5cc1a83-5420-4190-8e72-5eb49f523597-kube-api-access-2r4f4\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.692123 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.691997    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cc1a83-5420-4190-8e72-5eb49f523597-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.692123 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:04:41.692056    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-serving-cert: secret "maas-api-e2e-inf-b-fc9900-serving-cert" not found
Sep 09 07:04:41.692285 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:04:41.692127    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls podName:a5cc1a83-5420-4190-8e72-5eb49f523597 nodeName:}" failed. No retries permitted until 2026-09-09 07:04:42.192108017 +0000 UTC m=+814.562367098 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls") pod "maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2" (UID: "a5cc1a83-5420-4190-8e72-5eb49f523597") : secret "maas-api-e2e-inf-b-fc9900-serving-cert" not found
Sep 09 07:04:41.692285 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.692061    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-metrics-certs\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.692718 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.692696    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cc1a83-5420-4190-8e72-5eb49f523597-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.694465 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.694442    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-metrics-certs\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.701236 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.701210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2r4f4\" (UniqueName: \"kubernetes.io/projected/a5cc1a83-5420-4190-8e72-5eb49f523597-kube-api-access-2r4f4\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:41.980950 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:41.980907    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:42.076830 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:42.076801    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:42.196941 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:42.196912    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:42.199418 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:42.199381    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls\") pod \"maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:42.239394 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:42.239318    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:42.432652 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:42.432618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:42.708982 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:42.708956    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"]
Sep 09 07:04:42.710780 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:04:42.710736    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5cc1a83_5420_4190_8e72_5eb49f523597.slice/crio-751ab00cb02e43a49e0a797cd41d27836ef18048e3c247345a2f103edf3d3eb4 WatchSource:0}: Error finding container 751ab00cb02e43a49e0a797cd41d27836ef18048e3c247345a2f103edf3d3eb4: Status 404 returned error can't find the container with id 751ab00cb02e43a49e0a797cd41d27836ef18048e3c247345a2f103edf3d3eb4
Sep 09 07:04:42.745129 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:42.745101    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:43.005279 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:43.005192    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2" event={"ID":"a5cc1a83-5420-4190-8e72-5eb49f523597","Type":"ContainerStarted","Data":"751ab00cb02e43a49e0a797cd41d27836ef18048e3c247345a2f103edf3d3eb4"}
Sep 09 07:04:43.726059 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:43.726013    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:44.012089 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:44.012002    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2" event={"ID":"a5cc1a83-5420-4190-8e72-5eb49f523597","Type":"ContainerStarted","Data":"f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2"}
Sep 09 07:04:44.012240 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:44.012116    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:44.050448 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:44.050394    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2" podStartSLOduration=2.726247768 podStartE2EDuration="3.050379506s" totalImagesPullingTime="324.131738ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:04:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:04:44.04966594 +0000 UTC m=+816.419925031" watchObservedRunningTime="2026-09-09 07:04:44.050379506 +0000 UTC m=+816.420638625"
Sep 09 07:04:44.525870 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:44.525828    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:45.259922 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:45.259862    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:47.989191 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:47.989156    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:04:50.023276 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:50.023244    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:04:59.926654 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:04:59.926611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:00.464503 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.464464    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"]
Sep 09 07:05:00.464770 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.464736    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v" podUID="fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" containerName="maas-api" containerID="cri-o://f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0" gracePeriod=30
Sep 09 07:05:00.491673 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.491646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:00.557809 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.557771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:00.723689 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.723630    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:05:00.784531 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.784496    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-metrics-certs\") pod \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") "
Sep 09 07:05:00.784747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.784558    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-openshift-service-ca\") pod \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") "
Sep 09 07:05:00.784747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.784637    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls\") pod \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") "
Sep 09 07:05:00.784747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.784674    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vgtqm\" (UniqueName: \"kubernetes.io/projected/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-kube-api-access-vgtqm\") pod \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\" (UID: \"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c\") "
Sep 09 07:05:00.785149 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.784937    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" (UID: "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:05:00.785149 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.785050    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:05:00.787663 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.787629    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" (UID: "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:05:00.788740 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.788706    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" (UID: "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:05:00.788905 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.788874    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-kube-api-access-vgtqm" (OuterVolumeSpecName: "kube-api-access-vgtqm") pod "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" (UID: "fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c"). InnerVolumeSpecName "kube-api-access-vgtqm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:05:00.886092 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.886060    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:05:00.886092 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.886090    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:05:00.886092 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.886098    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vgtqm\" (UniqueName: \"kubernetes.io/projected/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c-kube-api-access-vgtqm\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:05:00.983891 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:00.983803    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:01.085087 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.085039    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:01.103165 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.103132    2563 generic.go:356] "Generic (PLEG): container finished" podID="fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" containerID="f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0" exitCode=0
Sep 09 07:05:01.103331 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.103177    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"
Sep 09 07:05:01.103331 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.103225    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v" event={"ID":"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c","Type":"ContainerDied","Data":"f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0"}
Sep 09 07:05:01.103331 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.103285    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v" event={"ID":"fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c","Type":"ContainerDied","Data":"5201bed8678555baa8c6ee442d0ee7d04261a1c47db1fb9687872d01159c6452"}
Sep 09 07:05:01.103331 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.103315    2563 scope.go:117] "RemoveContainer" containerID="f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0"
Sep 09 07:05:01.117258 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.117232    2563 scope.go:117] "RemoveContainer" containerID="f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0"
Sep 09 07:05:01.121142 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:05:01.121107    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0\": container with ID starting with f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0 not found: ID does not exist" containerID="f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0"
Sep 09 07:05:01.121328 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.121307    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0"} err="failed to get container status \"f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0\": rpc error: code = NotFound desc = could not find container \"f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0\": container with ID starting with f25b95b8278f4928a6d90429ebf09ec42c6571d2b07eba085bbf0684389f9bd0 not found: ID does not exist"
Sep 09 07:05:01.135010 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.134979    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"]
Sep 09 07:05:01.138323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.138301    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-ef07c382-5cbb4dd597-4dc2v"]
Sep 09 07:05:01.304829 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.304736    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:01.487749 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.487708    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:01.642920 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.642835    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:01.852721 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:01.852691    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:02.117382 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:02.117349    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" path="/var/lib/kubelet/pods/fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c/volumes"
Sep 09 07:05:02.367229 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:02.367195    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:02.729777 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:02.729728    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:04.933938 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:04.933904    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"]
Sep 09 07:05:04.934407 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:04.934392    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" containerName="maas-api"
Sep 09 07:05:04.934407 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:04.934407    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" containerName="maas-api"
Sep 09 07:05:04.934485 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:04.934481    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa4ebc4c-10e5-4d84-bd1a-1ebdefb8d37c" containerName="maas-api"
Sep 09 07:05:04.937667 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:04.937651    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:04.940207 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:04.940187    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-83162d4f-serving-cert\""
Sep 09 07:05:04.947686 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:04.947661    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"]
Sep 09 07:05:05.026026 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.025993    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.026190 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.026043    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls8zh\" (UniqueName: \"kubernetes.io/projected/fe3053a4-36d2-412b-8b35-0b194f99c33a-kube-api-access-ls8zh\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.026190 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.026087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-metrics-certs\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.026190 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.026132    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe3053a4-36d2-412b-8b35-0b194f99c33a-openshift-service-ca\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.126842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.126808    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.127012 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.126859    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls8zh\" (UniqueName: \"kubernetes.io/projected/fe3053a4-36d2-412b-8b35-0b194f99c33a-kube-api-access-ls8zh\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.127012 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.126879    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-metrics-certs\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.127012 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:05:05.126944    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-serving-cert: secret "maas-api-e2e-migrate-83162d4f-serving-cert" not found
Sep 09 07:05:05.127012 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.126986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe3053a4-36d2-412b-8b35-0b194f99c33a-openshift-service-ca\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.127012 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:05:05.127012    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls podName:fe3053a4-36d2-412b-8b35-0b194f99c33a nodeName:}" failed. No retries permitted until 2026-09-09 07:05:05.626996177 +0000 UTC m=+837.997255243 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls") pod "maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg" (UID: "fe3053a4-36d2-412b-8b35-0b194f99c33a") : secret "maas-api-e2e-migrate-83162d4f-serving-cert" not found
Sep 09 07:05:05.127634 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.127617    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe3053a4-36d2-412b-8b35-0b194f99c33a-openshift-service-ca\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.129200 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.129183    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-metrics-certs\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.136678 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.136652    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls8zh\" (UniqueName: \"kubernetes.io/projected/fe3053a4-36d2-412b-8b35-0b194f99c33a-kube-api-access-ls8zh\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.337387 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.337302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:05.438311 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.438275    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:05.540196 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.540162    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:05.633221 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.633127    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.635560 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.635536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls\") pod \"maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:05.644142 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.644119    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:05.849912 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:05.849870    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:06.140706 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:06.140670    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"]
Sep 09 07:05:06.144364 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:05:06.144331    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfe3053a4_36d2_412b_8b35_0b194f99c33a.slice/crio-8ca13fd1d27ce889058af976d6a4f8d32fdf6683da980e1391f85780b4e61606 WatchSource:0}: Error finding container 8ca13fd1d27ce889058af976d6a4f8d32fdf6683da980e1391f85780b4e61606: Status 404 returned error can't find the container with id 8ca13fd1d27ce889058af976d6a4f8d32fdf6683da980e1391f85780b4e61606
Sep 09 07:05:07.137036 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:07.137000    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg" event={"ID":"fe3053a4-36d2-412b-8b35-0b194f99c33a","Type":"ContainerStarted","Data":"249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75"}
Sep 09 07:05:07.137229 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:07.137087    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg" event={"ID":"fe3053a4-36d2-412b-8b35-0b194f99c33a","Type":"ContainerStarted","Data":"8ca13fd1d27ce889058af976d6a4f8d32fdf6683da980e1391f85780b4e61606"}
Sep 09 07:05:07.137229 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:07.137118    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:07.160846 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:07.160794    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg" podStartSLOduration=2.7831112559999998 podStartE2EDuration="3.16078058s" totalImagesPullingTime="377.669324ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:05:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:05:07.158687537 +0000 UTC m=+839.528946627" watchObservedRunningTime="2026-09-09 07:05:07.16078058 +0000 UTC m=+839.531039667"
Sep 09 07:05:13.147695 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:13.147666    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:34.753339 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:34.753293    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:36.391106 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.391070    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"]
Sep 09 07:05:36.391527 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.391391    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg" podUID="fe3053a4-36d2-412b-8b35-0b194f99c33a" containerName="maas-api" containerID="cri-o://249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75" gracePeriod=30
Sep 09 07:05:36.428136 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.428099    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:36.521561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.521525    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:36.600002 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.599970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:36.661461 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.661437    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:36.696756 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.696720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:36.740701 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.740670    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-metrics-certs\") pod \"fe3053a4-36d2-412b-8b35-0b194f99c33a\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") "
Sep 09 07:05:36.740871 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.740725    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ls8zh\" (UniqueName: \"kubernetes.io/projected/fe3053a4-36d2-412b-8b35-0b194f99c33a-kube-api-access-ls8zh\") pod \"fe3053a4-36d2-412b-8b35-0b194f99c33a\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") "
Sep 09 07:05:36.740871 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.740756    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls\") pod \"fe3053a4-36d2-412b-8b35-0b194f99c33a\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") "
Sep 09 07:05:36.740871 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.740867    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe3053a4-36d2-412b-8b35-0b194f99c33a-openshift-service-ca\") pod \"fe3053a4-36d2-412b-8b35-0b194f99c33a\" (UID: \"fe3053a4-36d2-412b-8b35-0b194f99c33a\") "
Sep 09 07:05:36.741310 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.741279    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fe3053a4-36d2-412b-8b35-0b194f99c33a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "fe3053a4-36d2-412b-8b35-0b194f99c33a" (UID: "fe3053a4-36d2-412b-8b35-0b194f99c33a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:05:36.742999 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.742971    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fe3053a4-36d2-412b-8b35-0b194f99c33a-kube-api-access-ls8zh" (OuterVolumeSpecName: "kube-api-access-ls8zh") pod "fe3053a4-36d2-412b-8b35-0b194f99c33a" (UID: "fe3053a4-36d2-412b-8b35-0b194f99c33a"). InnerVolumeSpecName "kube-api-access-ls8zh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:05:36.742999 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.742987    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "fe3053a4-36d2-412b-8b35-0b194f99c33a" (UID: "fe3053a4-36d2-412b-8b35-0b194f99c33a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:05:36.743113 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.743007    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "fe3053a4-36d2-412b-8b35-0b194f99c33a" (UID: "fe3053a4-36d2-412b-8b35-0b194f99c33a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:05:36.817427 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.817394    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:36.842135 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.842101    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fe3053a4-36d2-412b-8b35-0b194f99c33a-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:05:36.842135 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.842132    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:05:36.842328 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.842147    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ls8zh\" (UniqueName: \"kubernetes.io/projected/fe3053a4-36d2-412b-8b35-0b194f99c33a-kube-api-access-ls8zh\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:05:36.842328 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.842159    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fe3053a4-36d2-412b-8b35-0b194f99c33a-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:05:36.980275 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:36.980236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:37.305960 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.305866    2563 generic.go:356] "Generic (PLEG): container finished" podID="fe3053a4-36d2-412b-8b35-0b194f99c33a" containerID="249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75" exitCode=0
Sep 09 07:05:37.305960 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.305909    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"
Sep 09 07:05:37.305960 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.305954    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg" event={"ID":"fe3053a4-36d2-412b-8b35-0b194f99c33a","Type":"ContainerDied","Data":"249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75"}
Sep 09 07:05:37.306219 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.306008    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg" event={"ID":"fe3053a4-36d2-412b-8b35-0b194f99c33a","Type":"ContainerDied","Data":"8ca13fd1d27ce889058af976d6a4f8d32fdf6683da980e1391f85780b4e61606"}
Sep 09 07:05:37.306219 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.306036    2563 scope.go:117] "RemoveContainer" containerID="249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75"
Sep 09 07:05:37.311826 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.311802    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:37.316068 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.316052    2563 scope.go:117] "RemoveContainer" containerID="249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75"
Sep 09 07:05:37.316363 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:05:37.316344    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75\": container with ID starting with 249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75 not found: ID does not exist" containerID="249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75"
Sep 09 07:05:37.316422 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.316371    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75"} err="failed to get container status \"249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75\": rpc error: code = NotFound desc = could not find container \"249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75\": container with ID starting with 249dcea6eaff2b04a46f57bb6bb723ac7643eb083ffb450cd193e1154ff22c75 not found: ID does not exist"
Sep 09 07:05:37.330204 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.330182    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"]
Sep 09 07:05:37.334617 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.334580    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-83162d4f-7b8994677d-jrmxg"]
Sep 09 07:05:37.710457 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.710423    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"]
Sep 09 07:05:37.710916 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.710901    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fe3053a4-36d2-412b-8b35-0b194f99c33a" containerName="maas-api"
Sep 09 07:05:37.710961 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.710920    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="fe3053a4-36d2-412b-8b35-0b194f99c33a" containerName="maas-api"
Sep 09 07:05:37.711006 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.710996    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="fe3053a4-36d2-412b-8b35-0b194f99c33a" containerName="maas-api"
Sep 09 07:05:37.715633 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.715614    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.722401 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.722373    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-7eec132b-openshift-default-dockercfg-zx9nc\""
Sep 09 07:05:37.733254 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.733231    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"]
Sep 09 07:05:37.741639 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.741605    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:37.751783 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.751754    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-credential-socket\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.751894 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.751805    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-token\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.751894 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.751826    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-podinfo\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.751894 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.751856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-data\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.752121 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.752102    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9k4p9\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-kube-api-access-9k4p9\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.752173 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.752135    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-certs\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.752251 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.752234    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-socket\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.752310 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.752254    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-envoy\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.752310 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.752285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d09e58-d58b-455e-9e5b-b86dbec0d084-istiod-ca-cert\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853005 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.852969    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-data\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853175 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9k4p9\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-kube-api-access-9k4p9\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853175 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853059    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-certs\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853175 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-socket\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853175 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-envoy\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853175 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d09e58-d58b-455e-9e5b-b86dbec0d084-istiod-ca-cert\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853175 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853167    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-credential-socket\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853480 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853348    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-token\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853480 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853431    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-data\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853480 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853443    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-socket\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853696 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853443    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-podinfo\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853755 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853706    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-credential-socket\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853902 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853884    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-certs\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.853978 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.853959    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d09e58-d58b-455e-9e5b-b86dbec0d084-istiod-ca-cert\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.856066 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.856028    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-envoy\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.856262 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.856239    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-podinfo\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.864000 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.863975    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9k4p9\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-kube-api-access-9k4p9\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:37.864807 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:37.864790    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-token\") pod \"e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") " pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:38.026067 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:38.025974    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:38.109441 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:38.109398    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:38.120407 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:38.120371    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fe3053a4-36d2-412b-8b35-0b194f99c33a" path="/var/lib/kubelet/pods/fe3053a4-36d2-412b-8b35-0b194f99c33a/volumes"
Sep 09 07:05:38.320449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:38.320367    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"]
Sep 09 07:05:38.323961 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:05:38.323936    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d09e58_d58b_455e_9e5b_b86dbec0d084.slice/crio-56be2968a149b70c18e3959a849db6c3809903d42f76a7f94b44d0b0034567c0 WatchSource:0}: Error finding container 56be2968a149b70c18e3959a849db6c3809903d42f76a7f94b44d0b0034567c0: Status 404 returned error can't find the container with id 56be2968a149b70c18e3959a849db6c3809903d42f76a7f94b44d0b0034567c0
Sep 09 07:05:39.319393 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:39.319349    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" event={"ID":"16d09e58-d58b-455e-9e5b-b86dbec0d084","Type":"ContainerStarted","Data":"56be2968a149b70c18e3959a849db6c3809903d42f76a7f94b44d0b0034567c0"}
Sep 09 07:05:41.191048 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:41.191013    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:05:41.191354 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:41.191088    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:05:41.191354 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:41.191129    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:05:41.333177 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:41.333140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" event={"ID":"16d09e58-d58b-455e-9e5b-b86dbec0d084","Type":"ContainerStarted","Data":"b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2"}
Sep 09 07:05:41.361657 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:41.361576    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" podStartSLOduration=1.496834941 podStartE2EDuration="4.361558794s" totalImagesPullingTime="2.864723853s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:05:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:05:41.358654474 +0000 UTC m=+873.728913562" watchObservedRunningTime="2026-09-09 07:05:41.361558794 +0000 UTC m=+873.731817974"
Sep 09 07:05:42.026745 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:42.026707    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:42.031997 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:42.031972    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:42.338487 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:42.338403    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:42.339349 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:42.339332    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:05:42.741204 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:42.741171    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:42.909611 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:42.909553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:43.933790 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:43.933757    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"]
Sep 09 07:05:43.937730 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:43.937708    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:43.942855 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:43.942829    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7eec132b-serving-cert\""
Sep 09 07:05:43.952251 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:43.952220    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"]
Sep 09 07:05:44.008711 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.008675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:44.019948 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.019921    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-maas-api-tls\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.020082 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.019978    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4snl\" (UniqueName: \"kubernetes.io/projected/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-kube-api-access-b4snl\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.020082 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.020054    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-metrics-certs\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.020155 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.020127    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-openshift-service-ca\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.116623 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.116577    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:44.120576 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.120557    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-metrics-certs\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.120678 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.120642    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-openshift-service-ca\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.120678 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.120671    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-maas-api-tls\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.120772 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.120709    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4snl\" (UniqueName: \"kubernetes.io/projected/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-kube-api-access-b4snl\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.121391 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.121365    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-openshift-service-ca\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.123004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.122985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-metrics-certs\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.123167 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.123147    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-maas-api-tls\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.139206 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.139175    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4snl\" (UniqueName: \"kubernetes.io/projected/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-kube-api-access-b4snl\") pod \"maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.215051 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.214969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:44.248545 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.248518    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:44.561523 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.561442    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"]
Sep 09 07:05:44.565481 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:05:44.565453    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30ae8cf2_96da_4814_874e_cbd3e86b5dc8.slice/crio-bebf3db2689619c5c6555d3670dd969eacce270513fc4be24130b68bd0d82d0c WatchSource:0}: Error finding container bebf3db2689619c5c6555d3670dd969eacce270513fc4be24130b68bd0d82d0c: Status 404 returned error can't find the container with id bebf3db2689619c5c6555d3670dd969eacce270513fc4be24130b68bd0d82d0c
Sep 09 07:05:44.661545 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.661508    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:44.823644 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:44.823548    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:05:45.365776 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:45.365736    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm" event={"ID":"30ae8cf2-96da-4814-874e-cbd3e86b5dc8","Type":"ContainerStarted","Data":"8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446"}
Sep 09 07:05:45.366192 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:45.365799    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm" event={"ID":"30ae8cf2-96da-4814-874e-cbd3e86b5dc8","Type":"ContainerStarted","Data":"bebf3db2689619c5c6555d3670dd969eacce270513fc4be24130b68bd0d82d0c"}
Sep 09 07:05:45.366192 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:45.365826    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:45.407713 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:45.407656    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm" podStartSLOduration=2.022173035 podStartE2EDuration="2.40763805s" totalImagesPullingTime="385.465015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:05:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:05:45.406638364 +0000 UTC m=+877.776897451" watchObservedRunningTime="2026-09-09 07:05:45.40763805 +0000 UTC m=+877.777897195"
Sep 09 07:05:51.377502 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:51.377470    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:05:58.379739 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:58.379700    2563 patch_prober.go:28] interesting pod/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 09 07:05:58.380111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:58.379773    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 09 07:05:59.439714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:59.439679    2563 generic.go:356] "Generic (PLEG): container finished" podID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerID="b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2" exitCode=0
Sep 09 07:05:59.440228 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:59.439764    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" event={"ID":"16d09e58-d58b-455e-9e5b-b86dbec0d084","Type":"ContainerDied","Data":"b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2"}
Sep 09 07:05:59.440303 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:59.440283    2563 scope.go:117] "RemoveContainer" containerID="b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2"
Sep 09 07:05:59.441515 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:59.441480    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:05:59.441647 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:59.441566    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:05:59.441647 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:05:59.441641    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:06:00.447321 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:00.447282    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" event={"ID":"16d09e58-d58b-455e-9e5b-b86dbec0d084","Type":"ContainerStarted","Data":"314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688"}
Sep 09 07:06:01.026789 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:01.026742    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:06:01.027009 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:01.026986    2563 patch_prober.go:28] interesting pod/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 07:06:01.027065 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:01.027046    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 07:06:02.026498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:02.026465    2563 patch_prober.go:28] interesting pod/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 07:06:02.026897 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:02.026526    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 07:06:03.026990 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:03.026955    2563 patch_prober.go:28] interesting pod/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 07:06:03.027377 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:03.027013    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 07:06:04.026780 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:04.026739    2563 patch_prober.go:28] interesting pod/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 07:06:04.026950 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:04.026798    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 07:06:04.402405 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:04.402328    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:05.026886 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:05.026857    2563 patch_prober.go:28] interesting pod/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused" start-of-body=
Sep 09 07:06:05.027106 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:05.026904    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.50:15021/healthz/ready\": dial tcp 10.133.0.50:15021: connect: connection refused"
Sep 09 07:06:06.030135 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:06.030101    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:06:06.030658 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:06.030372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:06:06.031104 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:06.031083    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:06:06.230934 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:06.230902    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:07.480680 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:07.480640    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:08.139667 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:08.139635    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:06:08.139858 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:08.139715    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:06:08.574868 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:08.574833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:09.704474 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:09.704436    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:15.964103 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:15.964067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:16.736801 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:16.736763    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"]
Sep 09 07:06:16.736997 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:16.736976    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn" podUID="ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" containerName="maas-api" containerID="cri-o://2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035" gracePeriod=30
Sep 09 07:06:16.770677 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:16.770648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:16.858079 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:16.858051    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:16.945749 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:16.945711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:17.002772 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.002687    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:06:17.056900 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.056872    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-metrics-certs\") pod \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") "
Sep 09 07:06:17.057081 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.056962    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-openshift-service-ca\") pod \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") "
Sep 09 07:06:17.057081 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.056989    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls\") pod \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") "
Sep 09 07:06:17.057081 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.057055    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkbgp\" (UniqueName: \"kubernetes.io/projected/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-kube-api-access-hkbgp\") pod \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\" (UID: \"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b\") "
Sep 09 07:06:17.057415 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.057380    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" (UID: "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:06:17.059085 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.059055    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" (UID: "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:06:17.059170 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.059124    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-kube-api-access-hkbgp" (OuterVolumeSpecName: "kube-api-access-hkbgp") pod "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" (UID: "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b"). InnerVolumeSpecName "kube-api-access-hkbgp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:06:17.059224 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.059208    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" (UID: "ca70c1d5-01f3-4fd9-b0bf-69c4a425314b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:06:17.158014 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.157971    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:17.158014 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.158007    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:17.158014 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.158022    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkbgp\" (UniqueName: \"kubernetes.io/projected/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-kube-api-access-hkbgp\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:17.158246 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.158110    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:17.551240 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.551201    2563 generic.go:356] "Generic (PLEG): container finished" podID="ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" containerID="2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035" exitCode=0
Sep 09 07:06:17.551388 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.551250    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"
Sep 09 07:06:17.551388 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.551287    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn" event={"ID":"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b","Type":"ContainerDied","Data":"2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035"}
Sep 09 07:06:17.551388 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.551337    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn" event={"ID":"ca70c1d5-01f3-4fd9-b0bf-69c4a425314b","Type":"ContainerDied","Data":"73ecf6bcf9970dbca70e1f7f140e781bf5912044673e45d9ec0f0d613b339caf"}
Sep 09 07:06:17.551388 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.551358    2563 scope.go:117] "RemoveContainer" containerID="2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035"
Sep 09 07:06:17.560920 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.560899    2563 scope.go:117] "RemoveContainer" containerID="2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035"
Sep 09 07:06:17.561201 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:06:17.561183    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035\": container with ID starting with 2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035 not found: ID does not exist" containerID="2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035"
Sep 09 07:06:17.561272 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.561208    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035"} err="failed to get container status \"2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035\": rpc error: code = NotFound desc = could not find container \"2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035\": container with ID starting with 2932934808fc5bc3f6c758346ed27ed8778f52eab4118af1044fbccf7361e035 not found: ID does not exist"
Sep 09 07:06:17.576708 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.576683    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"]
Sep 09 07:06:17.584528 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.584504    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-40c98b-6bdc659744-28pgn"]
Sep 09 07:06:17.586404 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.586372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:17.719851 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:17.719821    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:18.115491 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:18.115456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:18.117774 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:18.117753    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" path="/var/lib/kubelet/pods/ca70c1d5-01f3-4fd9-b0bf-69c4a425314b/volumes"
Sep 09 07:06:20.816933 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:20.816893    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:22.488985 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:22.488948    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:22.631882 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:22.631851    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:24.234902 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:24.234872    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:25.058268 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.058228    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"]
Sep 09 07:06:25.058538 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.058483    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm" podUID="30ae8cf2-96da-4814-874e-cbd3e86b5dc8" containerName="maas-api" containerID="cri-o://8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446" gracePeriod=30
Sep 09 07:06:25.156730 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.156692    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:25.294874 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.294840    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:25.325794 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.325772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:06:25.435082 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.435048    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:25.436027 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.436008    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-maas-api-tls\") pod \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") "
Sep 09 07:06:25.436128 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.436115    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-metrics-certs\") pod \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") "
Sep 09 07:06:25.436167 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.436138    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-openshift-service-ca\") pod \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") "
Sep 09 07:06:25.436167 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.436160    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4snl\" (UniqueName: \"kubernetes.io/projected/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-kube-api-access-b4snl\") pod \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\" (UID: \"30ae8cf2-96da-4814-874e-cbd3e86b5dc8\") "
Sep 09 07:06:25.436503 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.436475    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "30ae8cf2-96da-4814-874e-cbd3e86b5dc8" (UID: "30ae8cf2-96da-4814-874e-cbd3e86b5dc8"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:06:25.438022 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.438001    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "30ae8cf2-96da-4814-874e-cbd3e86b5dc8" (UID: "30ae8cf2-96da-4814-874e-cbd3e86b5dc8"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:06:25.438202 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.438164    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-kube-api-access-b4snl" (OuterVolumeSpecName: "kube-api-access-b4snl") pod "30ae8cf2-96da-4814-874e-cbd3e86b5dc8" (UID: "30ae8cf2-96da-4814-874e-cbd3e86b5dc8"). InnerVolumeSpecName "kube-api-access-b4snl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:06:25.438310 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.438247    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "30ae8cf2-96da-4814-874e-cbd3e86b5dc8" (UID: "30ae8cf2-96da-4814-874e-cbd3e86b5dc8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:06:25.537321 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.537288    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:25.537321 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.537322    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:25.537522 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.537335    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4snl\" (UniqueName: \"kubernetes.io/projected/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-kube-api-access-b4snl\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:25.537522 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.537349    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/30ae8cf2-96da-4814-874e-cbd3e86b5dc8-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:25.605073 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.604995    2563 generic.go:356] "Generic (PLEG): container finished" podID="30ae8cf2-96da-4814-874e-cbd3e86b5dc8" containerID="8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446" exitCode=0
Sep 09 07:06:25.605073 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.605050    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"
Sep 09 07:06:25.605319 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.605092    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm" event={"ID":"30ae8cf2-96da-4814-874e-cbd3e86b5dc8","Type":"ContainerDied","Data":"8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446"}
Sep 09 07:06:25.605319 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.605156    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm" event={"ID":"30ae8cf2-96da-4814-874e-cbd3e86b5dc8","Type":"ContainerDied","Data":"bebf3db2689619c5c6555d3670dd969eacce270513fc4be24130b68bd0d82d0c"}
Sep 09 07:06:25.605319 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.605187    2563 scope.go:117] "RemoveContainer" containerID="8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446"
Sep 09 07:06:25.616265 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.616245    2563 scope.go:117] "RemoveContainer" containerID="8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446"
Sep 09 07:06:25.616555 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:06:25.616536    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446\": container with ID starting with 8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446 not found: ID does not exist" containerID="8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446"
Sep 09 07:06:25.616699 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.616562    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446"} err="failed to get container status \"8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446\": rpc error: code = NotFound desc = could not find container \"8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446\": container with ID starting with 8a55a30614ff222cb7d4889a952c68fcea783d6b58d637ad873bc18d26814446 not found: ID does not exist"
Sep 09 07:06:25.649991 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.649965    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"]
Sep 09 07:06:25.652354 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.652319    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:25.661505 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.661484    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7eec132b-86d55f74bc-mh4zm"]
Sep 09 07:06:25.802634 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:25.802568    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:26.117638 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:26.117571    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30ae8cf2-96da-4814-874e-cbd3e86b5dc8" path="/var/lib/kubelet/pods/30ae8cf2-96da-4814-874e-cbd3e86b5dc8/volumes"
Sep 09 07:06:26.895853 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:26.895817    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"]
Sep 09 07:06:26.896223 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:26.896028    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy" containerID="cri-o://314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688" gracePeriod=30
Sep 09 07:06:26.956162 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:26.956125    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:27.702644 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.702610    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"]
Sep 09 07:06:27.703045 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.703032    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" containerName="maas-api"
Sep 09 07:06:27.703104 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.703047    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" containerName="maas-api"
Sep 09 07:06:27.703104 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.703067    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30ae8cf2-96da-4814-874e-cbd3e86b5dc8" containerName="maas-api"
Sep 09 07:06:27.703104 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.703072    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="30ae8cf2-96da-4814-874e-cbd3e86b5dc8" containerName="maas-api"
Sep 09 07:06:27.703205 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.703147    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="30ae8cf2-96da-4814-874e-cbd3e86b5dc8" containerName="maas-api"
Sep 09 07:06:27.703205 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.703159    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="ca70c1d5-01f3-4fd9-b0bf-69c4a425314b" containerName="maas-api"
Sep 09 07:06:27.707488 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.707467    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.710302 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.710284    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-29013a28-openshift-default-dockercfg-ltslm\""
Sep 09 07:06:27.727019 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.726996    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"]
Sep 09 07:06:27.862235 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862191    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-socket\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.862235 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862235    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-data\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.862472 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862263    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-certs\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.862472 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862368    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istiod-ca-cert\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.862472 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862403    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8fgsw\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-kube-api-access-8fgsw\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.862472 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862434    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-token\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.862714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-envoy\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.862714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-podinfo\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.862714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.862579    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-credential-socket\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.963854 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.963778    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-socket\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.963854 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.963812    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-data\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.963854 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.963846    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-certs\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.963901    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istiod-ca-cert\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.963933    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8fgsw\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-kube-api-access-8fgsw\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.963970    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-token\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.964004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-envoy\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.964025    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-podinfo\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.964048    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-credential-socket\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.964195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-data\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.964279    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-socket\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.964435    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-certs\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964826 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.964503    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-credential-socket\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.964826 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.964620    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istiod-ca-cert\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.966182 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.966156    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-envoy\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.966458 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.966441    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-podinfo\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.973707 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.973688    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-token\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:27.975566 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:27.975547    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8fgsw\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-kube-api-access-8fgsw\") pod \"e2e-derive-29013a28-openshift-default-796f689db4-p6vd7\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") " pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:28.017640 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:28.017618    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:28.348099 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:28.347918    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"]
Sep 09 07:06:28.355794 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:28.355754    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:06:28.355926 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:28.355845    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:06:28.355926 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:28.355890    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:06:28.625714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:28.625626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" event={"ID":"a2f416a9-f3ef-4248-8856-36d3a22fecdd","Type":"ContainerStarted","Data":"e85fb4ef8185b5c5d95b33decc42cfa9de0218957d633628fb6199130c204087"}
Sep 09 07:06:28.625714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:28.625693    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" event={"ID":"a2f416a9-f3ef-4248-8856-36d3a22fecdd","Type":"ContainerStarted","Data":"27693ed84c9b62604f55db2706a50d3c4fcab2f811987f19c5780691675a8e09"}
Sep 09 07:06:28.654505 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:28.654436    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" podStartSLOduration=1.654413151 podStartE2EDuration="1.654413151s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:06:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:06:28.651372536 +0000 UTC m=+921.021631636" watchObservedRunningTime="2026-09-09 07:06:28.654413151 +0000 UTC m=+921.024672240"
Sep 09 07:06:29.018435 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:29.018396    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:29.024443 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:29.024416    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:29.630716 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:29.630685    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:29.631846 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:29.631827    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:29.728203 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:29.728154    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:30.271831 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:30.271796    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:30.380667 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:30.380576    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:31.284970 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.284938    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"]
Sep 09 07:06:31.289106 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.289083    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.292564 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.292540    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-29013a28-serving-cert\""
Sep 09 07:06:31.302668 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.302574    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"]
Sep 09 07:06:31.336478 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.336446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:31.398691 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.398656    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-metrics-certs\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.398858 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.398783    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-openshift-service-ca\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.398858 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.398833    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zndtj\" (UniqueName: \"kubernetes.io/projected/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-kube-api-access-zndtj\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.398937 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.398856    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.440330 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.440286    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:31.500188 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.500151    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-metrics-certs\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.500375 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.500238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-openshift-service-ca\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.500375 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.500264    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zndtj\" (UniqueName: \"kubernetes.io/projected/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-kube-api-access-zndtj\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.500375 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.500282    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.500560 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:06:31.500400    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-serving-cert: secret "maas-api-e2e-derive-29013a28-serving-cert" not found
Sep 09 07:06:31.500560 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:06:31.500475    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls podName:9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91 nodeName:}" failed. No retries permitted until 2026-09-09 07:06:32.000455275 +0000 UTC m=+924.370714342 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls") pod "maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz" (UID: "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91") : secret "maas-api-e2e-derive-29013a28-serving-cert" not found
Sep 09 07:06:31.501010 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.500985    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-openshift-service-ca\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.502749 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.502727    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-metrics-certs\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.511729 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.511703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zndtj\" (UniqueName: \"kubernetes.io/projected/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-kube-api-access-zndtj\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:31.567178 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.567069    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:31.960326 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:31.960266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:32.006505 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.006476    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:32.009054 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.008962    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls\") pod \"maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:32.081877 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.081838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:32.145825 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.145802    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:06:32.202012 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.201978    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.314920    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-envoy\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315001    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9k4p9\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-kube-api-access-9k4p9\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315041    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-data\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315077    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d09e58-d58b-455e-9e5b-b86dbec0d084-istiod-ca-cert\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315106    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-socket\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315156    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-podinfo\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315212    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-token\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315274    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-certs\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315315    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-credential-socket\") pod \"16d09e58-d58b-455e-9e5b-b86dbec0d084\" (UID: \"16d09e58-d58b-455e-9e5b-b86dbec0d084\") "
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315528    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:06:32.317416 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.315694    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.318522 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.318449    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16d09e58-d58b-455e-9e5b-b86dbec0d084-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:06:32.318522 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.318483    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-data" (OuterVolumeSpecName: "istio-data") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:06:32.318522 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.318505    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:06:32.318755 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.318527    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:06:32.321144 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.321119    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:06:32.321248 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.321138    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:06:32.321248 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.321153    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-kube-api-access-9k4p9" (OuterVolumeSpecName: "kube-api-access-9k4p9") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "kube-api-access-9k4p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:06:32.322278 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.322260    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-token" (OuterVolumeSpecName: "istio-token") pod "16d09e58-d58b-455e-9e5b-b86dbec0d084" (UID: "16d09e58-d58b-455e-9e5b-b86dbec0d084"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:06:32.417098 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.417065    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.417098 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.417093    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.417098 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.417103    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.417323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.417114    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9k4p9\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-kube-api-access-9k4p9\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.417323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.417122    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.417323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.417130    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d09e58-d58b-455e-9e5b-b86dbec0d084-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.417323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.417138    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.417323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.417146    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d09e58-d58b-455e-9e5b-b86dbec0d084-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:06:32.523543 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.523518    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"]
Sep 09 07:06:32.524463 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:06:32.524437    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9222ac9e_eb1d_41ea_a8ae_5aafcb7acd91.slice/crio-86823261a1de22dab5bab731b8deca8346b31242a144851d689de4127d6841e2 WatchSource:0}: Error finding container 86823261a1de22dab5bab731b8deca8346b31242a144851d689de4127d6841e2: Status 404 returned error can't find the container with id 86823261a1de22dab5bab731b8deca8346b31242a144851d689de4127d6841e2
Sep 09 07:06:32.655908 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.655878    2563 generic.go:356] "Generic (PLEG): container finished" podID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerID="314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688" exitCode=0
Sep 09 07:06:32.656113 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.655930    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"
Sep 09 07:06:32.656113 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.655963    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" event={"ID":"16d09e58-d58b-455e-9e5b-b86dbec0d084","Type":"ContainerDied","Data":"314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688"}
Sep 09 07:06:32.656113 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.656022    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn" event={"ID":"16d09e58-d58b-455e-9e5b-b86dbec0d084","Type":"ContainerDied","Data":"56be2968a149b70c18e3959a849db6c3809903d42f76a7f94b44d0b0034567c0"}
Sep 09 07:06:32.656113 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.656052    2563 scope.go:117] "RemoveContainer" containerID="314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688"
Sep 09 07:06:32.657518 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.657494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz" event={"ID":"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91","Type":"ContainerStarted","Data":"86823261a1de22dab5bab731b8deca8346b31242a144851d689de4127d6841e2"}
Sep 09 07:06:32.666146 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.666129    2563 scope.go:117] "RemoveContainer" containerID="b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2"
Sep 09 07:06:32.674564 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.674546    2563 scope.go:117] "RemoveContainer" containerID="314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688"
Sep 09 07:06:32.674871 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:06:32.674849    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688\": container with ID starting with 314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688 not found: ID does not exist" containerID="314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688"
Sep 09 07:06:32.674919 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.674881    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688"} err="failed to get container status \"314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688\": rpc error: code = NotFound desc = could not find container \"314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688\": container with ID starting with 314753ffe11f64c73271b3708e4e413614817cfe815bff66b4f434093f475688 not found: ID does not exist"
Sep 09 07:06:32.674919 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.674897    2563 scope.go:117] "RemoveContainer" containerID="b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2"
Sep 09 07:06:32.675148 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:06:32.675128    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2\": container with ID starting with b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2 not found: ID does not exist" containerID="b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2"
Sep 09 07:06:32.675189 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.675154    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2"} err="failed to get container status \"b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2\": rpc error: code = NotFound desc = could not find container \"b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2\": container with ID starting with b5c738253e156356a2385ec1d1cd5d8b97d90a53dbb42913b5d7978bb6e77eb2 not found: ID does not exist"
Sep 09 07:06:32.692174 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.692153    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"]
Sep 09 07:06:32.699783 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:32.699760    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-7eec132b-openshift-default-5c8d97d7fb-jb8hn"]
Sep 09 07:06:33.665394 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:33.665353    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz" event={"ID":"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91","Type":"ContainerStarted","Data":"a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b"}
Sep 09 07:06:33.665801 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:33.665455    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:33.695403 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:33.695355    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz" podStartSLOduration=2.28805448 podStartE2EDuration="2.69533897s" totalImagesPullingTime="407.28449ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:06:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:06:33.692501586 +0000 UTC m=+926.062760673" watchObservedRunningTime="2026-09-09 07:06:33.69533897 +0000 UTC m=+926.065598057"
Sep 09 07:06:34.117598 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:34.117549    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" path="/var/lib/kubelet/pods/16d09e58-d58b-455e-9e5b-b86dbec0d084/volumes"
Sep 09 07:06:39.677628 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:39.677577    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:06:50.352674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:50.352632    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:52.777829 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:52.777795    2563 generic.go:356] "Generic (PLEG): container finished" podID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerID="e85fb4ef8185b5c5d95b33decc42cfa9de0218957d633628fb6199130c204087" exitCode=0
Sep 09 07:06:52.778352 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:52.777872    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" event={"ID":"a2f416a9-f3ef-4248-8856-36d3a22fecdd","Type":"ContainerDied","Data":"e85fb4ef8185b5c5d95b33decc42cfa9de0218957d633628fb6199130c204087"}
Sep 09 07:06:52.778429 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:52.778414    2563 scope.go:117] "RemoveContainer" containerID="e85fb4ef8185b5c5d95b33decc42cfa9de0218957d633628fb6199130c204087"
Sep 09 07:06:52.779534 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:52.779506    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:06:52.779653 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:52.779612    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:06:52.779724 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:52.779665    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:06:53.784654 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:53.784615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" event={"ID":"a2f416a9-f3ef-4248-8856-36d3a22fecdd","Type":"ContainerStarted","Data":"210d6f9ec005690973e6d9c3bb8948d25e3541469a23bed4d1de0103fdd57dc3"}
Sep 09 07:06:54.018688 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:54.018653    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:54.018943 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:54.018918    2563 patch_prober.go:28] interesting pod/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 09 07:06:54.019030 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:54.018989    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 09 07:06:55.018563 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:55.018521    2563 patch_prober.go:28] interesting pod/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 09 07:06:55.019034 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:55.018580    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 09 07:06:56.018175 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:56.018136    2563 patch_prober.go:28] interesting pod/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 09 07:06:56.018356 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:56.018208    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 09 07:06:57.018378 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:57.018339    2563 patch_prober.go:28] interesting pod/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 09 07:06:57.018793 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:57.018408    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 09 07:06:58.018367 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:58.018329    2563 patch_prober.go:28] interesting pod/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body=
Sep 09 07:06:58.018555 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:58.018388    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused"
Sep 09 07:06:59.022766 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:59.022731    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:59.023188 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:59.023010    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:59.023758 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:59.023737    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:06:59.637799 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:59.637765    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:06:59.900234 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:06:59.900136    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:00.350912 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.350867    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:00.650427 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.650351    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"]
Sep 09 07:07:00.650899 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.650871    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy"
Sep 09 07:07:00.650971 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.650900    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy"
Sep 09 07:07:00.651059 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.651046    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy"
Sep 09 07:07:00.651143 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.651088    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy"
Sep 09 07:07:00.653246 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.653228    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.656756 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.656732    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-a1a33e-openshift-default-dockercfg-vmn5p\""
Sep 09 07:07:00.667895 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.667871    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"]
Sep 09 07:07:00.672693 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.672672    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:00.731031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.730991    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-istio-token\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.731252 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.731049    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-credential-socket\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.731252 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.731171    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-socket\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.731252 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.731210    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dtbb\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-kube-api-access-9dtbb\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.731252 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.731246    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-data\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.731473 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.731281    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-certs\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.731473 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.731314    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663936c-b2d0-4e74-a46e-8b1417086440-istiod-ca-cert\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.731473 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.731385    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663936c-b2d0-4e74-a46e-8b1417086440-istio-podinfo\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.731675 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.731496    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-envoy\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.816202 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.816169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:00.832962 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.832930    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-istio-token\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833140 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.832976    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-credential-socket\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833140 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833027    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-socket\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833140 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833043    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dtbb\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-kube-api-access-9dtbb\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833140 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833064    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-data\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833140 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833093    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-certs\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833140 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833131    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663936c-b2d0-4e74-a46e-8b1417086440-istiod-ca-cert\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833475 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833202    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663936c-b2d0-4e74-a46e-8b1417086440-istio-podinfo\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833475 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833238    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-envoy\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833475 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833438    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-socket\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833475 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-credential-socket\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833732 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-certs\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.833835 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.833813    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-data\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.834478 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.834450    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663936c-b2d0-4e74-a46e-8b1417086440-istiod-ca-cert\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.836047 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.836023    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-envoy\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.836273 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.836252    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663936c-b2d0-4e74-a46e-8b1417086440-istio-podinfo\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.844795 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.844773    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-istio-token\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.844998 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.844976    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dtbb\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-kube-api-access-9dtbb\") pod \"e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") " pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:00.963634 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:00.963571    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:01.240389 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:01.240359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"]
Sep 09 07:07:01.242769 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:07:01.242742    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9663936c_b2d0_4e74_a46e_8b1417086440.slice/crio-69e47abe5515e64c22e4e6072dc100f91b858ed710b913a1c74e4328cd7c1de6 WatchSource:0}: Error finding container 69e47abe5515e64c22e4e6072dc100f91b858ed710b913a1c74e4328cd7c1de6: Status 404 returned error can't find the container with id 69e47abe5515e64c22e4e6072dc100f91b858ed710b913a1c74e4328cd7c1de6
Sep 09 07:07:01.245021 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:01.244992    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:07:01.245085 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:01.245058    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:07:01.245131 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:01.245086    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:07:01.831804 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:01.831766    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" event={"ID":"9663936c-b2d0-4e74-a46e-8b1417086440","Type":"ContainerStarted","Data":"f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f"}
Sep 09 07:07:01.832165 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:01.831832    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" event={"ID":"9663936c-b2d0-4e74-a46e-8b1417086440","Type":"ContainerStarted","Data":"69e47abe5515e64c22e4e6072dc100f91b858ed710b913a1c74e4328cd7c1de6"}
Sep 09 07:07:01.860614 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:01.860554    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" podStartSLOduration=1.8605382000000001 podStartE2EDuration="1.8605382s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:07:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:07:01.859241007 +0000 UTC m=+954.229500121" watchObservedRunningTime="2026-09-09 07:07:01.8605382 +0000 UTC m=+954.230797287"
Sep 09 07:07:01.964644 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:01.964611    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:02.129407 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.129329    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"]
Sep 09 07:07:02.129563 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.129541    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz" podUID="9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" containerName="maas-api" containerID="cri-o://a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b" gracePeriod=30
Sep 09 07:07:02.193737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.193701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:02.387177 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.387112    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:07:02.451569 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.451534    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-metrics-certs\") pod \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") "
Sep 09 07:07:02.451764 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.451631    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zndtj\" (UniqueName: \"kubernetes.io/projected/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-kube-api-access-zndtj\") pod \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") "
Sep 09 07:07:02.451764 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.451716    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls\") pod \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") "
Sep 09 07:07:02.451865 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.451767    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-openshift-service-ca\") pod \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\" (UID: \"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91\") "
Sep 09 07:07:02.452190 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.452161    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" (UID: "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:07:02.453934 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.453908    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" (UID: "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:07:02.454039 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.453964    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" (UID: "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:07:02.454039 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.454000    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-kube-api-access-zndtj" (OuterVolumeSpecName: "kube-api-access-zndtj") pod "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" (UID: "9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91"). InnerVolumeSpecName "kube-api-access-zndtj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:07:02.552829 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.552791    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:02.552829 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.552829    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zndtj\" (UniqueName: \"kubernetes.io/projected/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-kube-api-access-zndtj\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:02.553061 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.552848    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:02.553061 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.552862    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:02.594414 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.594380    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"]
Sep 09 07:07:02.594670 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.594637    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy" containerID="cri-o://210d6f9ec005690973e6d9c3bb8948d25e3541469a23bed4d1de0103fdd57dc3" gracePeriod=30
Sep 09 07:07:02.838327 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.838287    2563 generic.go:356] "Generic (PLEG): container finished" podID="9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" containerID="a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b" exitCode=0
Sep 09 07:07:02.838728 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.838344    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"
Sep 09 07:07:02.838728 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.838368    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz" event={"ID":"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91","Type":"ContainerDied","Data":"a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b"}
Sep 09 07:07:02.838728 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.838446    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz" event={"ID":"9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91","Type":"ContainerDied","Data":"86823261a1de22dab5bab731b8deca8346b31242a144851d689de4127d6841e2"}
Sep 09 07:07:02.838728 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.838474    2563 scope.go:117] "RemoveContainer" containerID="a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b"
Sep 09 07:07:02.850704 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.850680    2563 scope.go:117] "RemoveContainer" containerID="a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b"
Sep 09 07:07:02.851140 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:07:02.851118    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b\": container with ID starting with a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b not found: ID does not exist" containerID="a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b"
Sep 09 07:07:02.851214 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.851151    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b"} err="failed to get container status \"a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b\": rpc error: code = NotFound desc = could not find container \"a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b\": container with ID starting with a32ca87c2f4d13220a526c3212e46670262d7faaec187cd614d9c5567d56030b not found: ID does not exist"
Sep 09 07:07:02.861663 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.861632    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"]
Sep 09 07:07:02.865746 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.865726    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-29013a28-7bfd77bd98-8qpjz"]
Sep 09 07:07:02.921209 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.921177    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:02.970096 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:02.970064    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:03.001760 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:03.001734    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:03.304664 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:03.304634    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:03.406484 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:03.406446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:03.531186 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:03.531150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:03.839350 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:03.839313    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:03.845260 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:03.845227    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:03.846452 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:03.846431    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:04.027905 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.027873    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:04.117976 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.117899    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" path="/var/lib/kubelet/pods/9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91/volumes"
Sep 09 07:07:04.949827 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.949796    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"]
Sep 09 07:07:04.950349 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.950328    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" containerName="maas-api"
Sep 09 07:07:04.950425 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.950384    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" containerName="maas-api"
Sep 09 07:07:04.950425 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.950416    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy"
Sep 09 07:07:04.950425 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.950425    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d09e58-d58b-455e-9e5b-b86dbec0d084" containerName="istio-proxy"
Sep 09 07:07:04.950778 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.950664    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9222ac9e-eb1d-41ea-a8ae-5aafcb7acd91" containerName="maas-api"
Sep 09 07:07:04.954119 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.954101    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:04.959206 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.959184    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-a1a33e-serving-cert\""
Sep 09 07:07:04.970467 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.970442    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"]
Sep 09 07:07:04.985244 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:04.985212    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:05.077358 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.077323    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-openshift-service-ca\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.077555 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.077371    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-maas-api-tls\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.077555 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.077432    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r25d6\" (UniqueName: \"kubernetes.io/projected/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-kube-api-access-r25d6\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.077555 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.077462    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-metrics-certs\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.085481 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.085455    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:05.178254 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.178178    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-openshift-service-ca\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.178254 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.178253    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-maas-api-tls\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.178532 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.178326    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r25d6\" (UniqueName: \"kubernetes.io/projected/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-kube-api-access-r25d6\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.178532 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.178388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-metrics-certs\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.179288 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.179256    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-openshift-service-ca\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.180887 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.180862    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-metrics-certs\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.181025 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.181004    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-maas-api-tls\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.184121 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.184078    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:05.186846 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.186828    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r25d6\" (UniqueName: \"kubernetes.io/projected/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-kube-api-access-r25d6\") pod \"maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.265621 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.265507    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:05.495813 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.495776    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:05.552348 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.552325    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"]
Sep 09 07:07:05.554513 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:07:05.554484    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd7eb0af_40bb_4d61_b625_5c10b9bbcc39.slice/crio-cdd3f00d2061cb0fc337c7557aff5a16b215ba6afc8e640e94b5007a899c712a WatchSource:0}: Error finding container cdd3f00d2061cb0fc337c7557aff5a16b215ba6afc8e640e94b5007a899c712a: Status 404 returned error can't find the container with id cdd3f00d2061cb0fc337c7557aff5a16b215ba6afc8e640e94b5007a899c712a
Sep 09 07:07:05.555990 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.555966    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 07:07:05.600454 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.600423    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:05.860010 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:05.859920    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml" event={"ID":"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39","Type":"ContainerStarted","Data":"cdd3f00d2061cb0fc337c7557aff5a16b215ba6afc8e640e94b5007a899c712a"}
Sep 09 07:07:06.866518 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:06.866480    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml" event={"ID":"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39","Type":"ContainerStarted","Data":"17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7"}
Sep 09 07:07:06.866968 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:06.866610    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:06.894228 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:06.894180    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml" podStartSLOduration=2.5352885929999998 podStartE2EDuration="2.894163788s" totalImagesPullingTime="358.875195ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:07:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:07:06.893854605 +0000 UTC m=+959.264113696" watchObservedRunningTime="2026-09-09 07:07:06.894163788 +0000 UTC m=+959.264422876"
Sep 09 07:07:07.874776 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:07.874740    2563 generic.go:356] "Generic (PLEG): container finished" podID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerID="210d6f9ec005690973e6d9c3bb8948d25e3541469a23bed4d1de0103fdd57dc3" exitCode=0
Sep 09 07:07:07.875138 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:07.874806    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" event={"ID":"a2f416a9-f3ef-4248-8856-36d3a22fecdd","Type":"ContainerDied","Data":"210d6f9ec005690973e6d9c3bb8948d25e3541469a23bed4d1de0103fdd57dc3"}
Sep 09 07:07:07.875138 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:07.874896    2563 scope.go:117] "RemoveContainer" containerID="e85fb4ef8185b5c5d95b33decc42cfa9de0218957d633628fb6199130c204087"
Sep 09 07:07:07.951433 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:07.951405    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:07:08.107174 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107090    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-podinfo\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107174 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107124    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-credential-socket\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107174 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107145    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-token\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107456 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107226    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-data\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107456 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107248    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-envoy\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107456 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107273    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-socket\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107456 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107303    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8fgsw\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-kube-api-access-8fgsw\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107456 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107335    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-certs\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107773 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107459    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:07:08.107773 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107517    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-data" (OuterVolumeSpecName: "istio-data") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:07:08.107773 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107613    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:07:08.107773 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107669    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istiod-ca-cert\") pod \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\" (UID: \"a2f416a9-f3ef-4248-8856-36d3a22fecdd\") "
Sep 09 07:07:08.107773 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.107749    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:07:08.108110 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.108017    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:07:08.108178 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.108142    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.108178 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.108162    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.108292 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.108177    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.108292 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.108191    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.108292 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.108205    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.109401 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.109372    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:07:08.109514 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.109413    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:07:08.109514 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.109414    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-kube-api-access-8fgsw" (OuterVolumeSpecName: "kube-api-access-8fgsw") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "kube-api-access-8fgsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:07:08.109743 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.109719    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-token" (OuterVolumeSpecName: "istio-token") pod "a2f416a9-f3ef-4248-8856-36d3a22fecdd" (UID: "a2f416a9-f3ef-4248-8856-36d3a22fecdd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:07:08.208966 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.208940    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8fgsw\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-kube-api-access-8fgsw\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.209100 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.209084    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.209210 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.209193    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.209210 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.209215    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a2f416a9-f3ef-4248-8856-36d3a22fecdd-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:07:08.882805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.882772    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"
Sep 09 07:07:08.882805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.882796    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7" event={"ID":"a2f416a9-f3ef-4248-8856-36d3a22fecdd","Type":"ContainerDied","Data":"27693ed84c9b62604f55db2706a50d3c4fcab2f811987f19c5780691675a8e09"}
Sep 09 07:07:08.883307 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.882867    2563 scope.go:117] "RemoveContainer" containerID="210d6f9ec005690973e6d9c3bb8948d25e3541469a23bed4d1de0103fdd57dc3"
Sep 09 07:07:08.901351 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.901328    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"]
Sep 09 07:07:08.906164 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:08.906122    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-29013a28-openshift-default-796f689db4-p6vd7"]
Sep 09 07:07:10.125933 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:10.125901    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" path="/var/lib/kubelet/pods/a2f416a9-f3ef-4248-8856-36d3a22fecdd/volumes"
Sep 09 07:07:11.120031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:11.119993    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:11.184967 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:11.184932    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:11.276744 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:11.276712    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:11.376862 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:11.376778    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:11.477996 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:11.477964    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:11.823133 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:11.823104    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:12.880102 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:12.880074    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:07:23.975685 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:23.975652    2563 generic.go:356] "Generic (PLEG): container finished" podID="9663936c-b2d0-4e74-a46e-8b1417086440" containerID="f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f" exitCode=0
Sep 09 07:07:23.976205 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:23.975729    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" event={"ID":"9663936c-b2d0-4e74-a46e-8b1417086440","Type":"ContainerDied","Data":"f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f"}
Sep 09 07:07:23.976277 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:23.976206    2563 scope.go:117] "RemoveContainer" containerID="f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f"
Sep 09 07:07:23.977271 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:23.977242    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:07:23.977369 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:23.977306    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:07:23.977369 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:23.977337    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:07:24.983176 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:24.983125    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" event={"ID":"9663936c-b2d0-4e74-a46e-8b1417086440","Type":"ContainerStarted","Data":"0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8"}
Sep 09 07:07:25.964489 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:25.964445    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:25.964727 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:25.964604    2563 patch_prober.go:28] interesting pod/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 09 07:07:25.964727 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:25.964655    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 09 07:07:26.964830 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:26.964792    2563 patch_prober.go:28] interesting pod/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 09 07:07:26.965394 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:26.965362    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 09 07:07:27.965031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:27.964993    2563 patch_prober.go:28] interesting pod/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 09 07:07:27.965657 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:27.965623    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 09 07:07:28.780442 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:28.780408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:28.965042 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:28.965001    2563 patch_prober.go:28] interesting pod/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body=
Sep 09 07:07:28.965637 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:28.965579    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused"
Sep 09 07:07:30.178507 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:30.178471    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:30.179045 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:30.178904    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:30.181044 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:30.181019    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:07:37.793767 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:37.793724    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:43.926789 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:43.926750    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:44.038865 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:44.038833    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:44.186505 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:44.186416    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:48.861332 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:48.861299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:52.240722 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:52.240690    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:56.139569 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:56.139536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:57.381534 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:57.381501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:07:58.280892 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:07:58.280856    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:00.065220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:00.065189    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:02.512811 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:02.512778    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:03.126910 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.126876    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"]
Sep 09 07:08:03.127159 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.127126    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml" podUID="bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" containerName="maas-api" containerID="cri-o://17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7" gracePeriod=30
Sep 09 07:08:03.157686 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.157659    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:03.242191 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.242154    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:03.335431 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.335399    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:03.385630 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.385541    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:08:03.423727 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.423695    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:03.467730 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.467697    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r25d6\" (UniqueName: \"kubernetes.io/projected/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-kube-api-access-r25d6\") pod \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") "
Sep 09 07:08:03.467910 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.467768    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-openshift-service-ca\") pod \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") "
Sep 09 07:08:03.467910 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.467847    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-metrics-certs\") pod \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") "
Sep 09 07:08:03.467910 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.467896    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-maas-api-tls\") pod \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\" (UID: \"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39\") "
Sep 09 07:08:03.468174 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.468150    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" (UID: "bd7eb0af-40bb-4d61-b625-5c10b9bbcc39"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:08:03.468316 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.468300    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:03.469872 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.469838    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-kube-api-access-r25d6" (OuterVolumeSpecName: "kube-api-access-r25d6") pod "bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" (UID: "bd7eb0af-40bb-4d61-b625-5c10b9bbcc39"). InnerVolumeSpecName "kube-api-access-r25d6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:08:03.469982 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.469883    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" (UID: "bd7eb0af-40bb-4d61-b625-5c10b9bbcc39"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:08:03.470281 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.470266    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" (UID: "bd7eb0af-40bb-4d61-b625-5c10b9bbcc39"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:08:03.502750 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.502722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:03.569726 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.569699    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r25d6\" (UniqueName: \"kubernetes.io/projected/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-kube-api-access-r25d6\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:03.569726 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.569723    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:03.569726 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.569732    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:03.788674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:03.788646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:04.197930 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.197895    2563 generic.go:356] "Generic (PLEG): container finished" podID="bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" containerID="17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7" exitCode=0
Sep 09 07:08:04.197930 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.197932    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml" event={"ID":"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39","Type":"ContainerDied","Data":"17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7"}
Sep 09 07:08:04.198180 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.197942    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"
Sep 09 07:08:04.198180 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.197972    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml" event={"ID":"bd7eb0af-40bb-4d61-b625-5c10b9bbcc39","Type":"ContainerDied","Data":"cdd3f00d2061cb0fc337c7557aff5a16b215ba6afc8e640e94b5007a899c712a"}
Sep 09 07:08:04.198180 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.197996    2563 scope.go:117] "RemoveContainer" containerID="17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7"
Sep 09 07:08:04.207950 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.207928    2563 scope.go:117] "RemoveContainer" containerID="17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7"
Sep 09 07:08:04.208254 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:08:04.208233    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7\": container with ID starting with 17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7 not found: ID does not exist" containerID="17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7"
Sep 09 07:08:04.208337 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.208262    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7"} err="failed to get container status \"17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7\": rpc error: code = NotFound desc = could not find container \"17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7\": container with ID starting with 17ec62ccca38966c25cf23865b7ba526768eccf355738446f3d3c245ad5428f7 not found: ID does not exist"
Sep 09 07:08:04.218272 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.218233    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"]
Sep 09 07:08:04.220736 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:04.220714    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-a1a33e-bc5db9659-xh7ml"]
Sep 09 07:08:06.117811 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:06.117778    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" path="/var/lib/kubelet/pods/bd7eb0af-40bb-4d61-b625-5c10b9bbcc39/volumes"
Sep 09 07:08:06.496771 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:06.496735    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:06.601048 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:06.600995    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:06.697832 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:06.697795    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:07.602043 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:07.602007    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:08.225019 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.224990    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"]
Sep 09 07:08:08.225266 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.225219    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f" podUID="f0033853-68db-42f6-8da6-41b9d1d642c3" containerName="maas-api" containerID="cri-o://1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3" gracePeriod=30
Sep 09 07:08:08.484860 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.484799    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:08:08.619741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.619710    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-metrics-certs\") pod \"f0033853-68db-42f6-8da6-41b9d1d642c3\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") "
Sep 09 07:08:08.620251 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.619786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6xx26\" (UniqueName: \"kubernetes.io/projected/f0033853-68db-42f6-8da6-41b9d1d642c3-kube-api-access-6xx26\") pod \"f0033853-68db-42f6-8da6-41b9d1d642c3\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") "
Sep 09 07:08:08.620251 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.620072    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0033853-68db-42f6-8da6-41b9d1d642c3-openshift-service-ca\") pod \"f0033853-68db-42f6-8da6-41b9d1d642c3\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") "
Sep 09 07:08:08.620251 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.620165    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-maas-api-tls\") pod \"f0033853-68db-42f6-8da6-41b9d1d642c3\" (UID: \"f0033853-68db-42f6-8da6-41b9d1d642c3\") "
Sep 09 07:08:08.620455 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.620429    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f0033853-68db-42f6-8da6-41b9d1d642c3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f0033853-68db-42f6-8da6-41b9d1d642c3" (UID: "f0033853-68db-42f6-8da6-41b9d1d642c3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:08:08.620574 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.620547    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f0033853-68db-42f6-8da6-41b9d1d642c3-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:08.622440 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.622411    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f0033853-68db-42f6-8da6-41b9d1d642c3" (UID: "f0033853-68db-42f6-8da6-41b9d1d642c3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:08:08.622625 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.622582    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f0033853-68db-42f6-8da6-41b9d1d642c3" (UID: "f0033853-68db-42f6-8da6-41b9d1d642c3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:08:08.622702 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.622621    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0033853-68db-42f6-8da6-41b9d1d642c3-kube-api-access-6xx26" (OuterVolumeSpecName: "kube-api-access-6xx26") pod "f0033853-68db-42f6-8da6-41b9d1d642c3" (UID: "f0033853-68db-42f6-8da6-41b9d1d642c3"). InnerVolumeSpecName "kube-api-access-6xx26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:08:08.721425 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.721392    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:08.721425 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.721420    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f0033853-68db-42f6-8da6-41b9d1d642c3-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:08.721425 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:08.721429    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6xx26\" (UniqueName: \"kubernetes.io/projected/f0033853-68db-42f6-8da6-41b9d1d642c3-kube-api-access-6xx26\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:09.091254 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.091214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:09.191580 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.191542    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:09.229360 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.229331    2563 generic.go:356] "Generic (PLEG): container finished" podID="f0033853-68db-42f6-8da6-41b9d1d642c3" containerID="1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3" exitCode=0
Sep 09 07:08:09.229562 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.229386    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"
Sep 09 07:08:09.229562 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.229427    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f" event={"ID":"f0033853-68db-42f6-8da6-41b9d1d642c3","Type":"ContainerDied","Data":"1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3"}
Sep 09 07:08:09.229562 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.229483    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f" event={"ID":"f0033853-68db-42f6-8da6-41b9d1d642c3","Type":"ContainerDied","Data":"cd59d59f46a43db6a8809cb08c00c7030e10f27bb06ba2963ce23dc6b759e79f"}
Sep 09 07:08:09.229562 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.229508    2563 scope.go:117] "RemoveContainer" containerID="1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3"
Sep 09 07:08:09.239391 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.239376    2563 scope.go:117] "RemoveContainer" containerID="1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3"
Sep 09 07:08:09.239674 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:08:09.239657    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3\": container with ID starting with 1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3 not found: ID does not exist" containerID="1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3"
Sep 09 07:08:09.239723 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.239682    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3"} err="failed to get container status \"1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3\": rpc error: code = NotFound desc = could not find container \"1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3\": container with ID starting with 1102e2b47f2fca83bfd2c5a4b613165ad277783d6f04b8d1bf7fb2110ff23ce3 not found: ID does not exist"
Sep 09 07:08:09.255611 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.255569    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"]
Sep 09 07:08:09.263878 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:09.263857    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-b212cb-777c46745d-wlh2f"]
Sep 09 07:08:10.119495 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:10.119465    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0033853-68db-42f6-8da6-41b9d1d642c3" path="/var/lib/kubelet/pods/f0033853-68db-42f6-8da6-41b9d1d642c3/volumes"
Sep 09 07:08:10.359812 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:10.359781    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:10.433034 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:10.432998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:10.631116 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:10.631079    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"]
Sep 09 07:08:10.631686 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:10.631628    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy" containerID="cri-o://0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8" gracePeriod=30
Sep 09 07:08:10.647941 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:10.647913    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:11.118981 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.118950    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:11.407337 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407245    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"]
Sep 09 07:08:11.407906 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407887    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" containerName="maas-api"
Sep 09 07:08:11.408004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407908    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" containerName="maas-api"
Sep 09 07:08:11.408004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407949    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0033853-68db-42f6-8da6-41b9d1d642c3" containerName="maas-api"
Sep 09 07:08:11.408004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407958    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0033853-68db-42f6-8da6-41b9d1d642c3" containerName="maas-api"
Sep 09 07:08:11.408004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407969    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy"
Sep 09 07:08:11.408004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407977    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy"
Sep 09 07:08:11.408004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407988    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy"
Sep 09 07:08:11.408004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.407996    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy"
Sep 09 07:08:11.408333 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.408100    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd7eb0af-40bb-4d61-b625-5c10b9bbcc39" containerName="maas-api"
Sep 09 07:08:11.408333 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.408113    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy"
Sep 09 07:08:11.408333 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.408131    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0033853-68db-42f6-8da6-41b9d1d642c3" containerName="maas-api"
Sep 09 07:08:11.408333 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.408141    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2f416a9-f3ef-4248-8856-36d3a22fecdd" containerName="istio-proxy"
Sep 09 07:08:11.412703 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.412682    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.415579 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.415557    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-43ead3-openshift-default-dockercfg-jgzfd\""
Sep 09 07:08:11.427469 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.427449    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"]
Sep 09 07:08:11.429854 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.429832    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:11.549241 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549202    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-podinfo\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.549241 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549240    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istiod-ca-cert\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.549477 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549285    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-data\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.549477 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549326    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-token\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.549477 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549389    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-envoy\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.549477 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549447    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-credential-socket\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.549676 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549480    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r8xf\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-kube-api-access-6r8xf\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.549676 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549502    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-certs\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.549676 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.549524    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-socket\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.650531 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650494    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-credential-socket\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.650531 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650550    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r8xf\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-kube-api-access-6r8xf\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.650839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-certs\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.650839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650640    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-socket\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.650839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650724    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-podinfo\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.650839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650749    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istiod-ca-cert\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.650839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650795    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-data\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.651099 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650852    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-token\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.651099 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650890    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-envoy\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.651099 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.650992    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-credential-socket\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.651099 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.651047    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-certs\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.651099 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.651082    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-socket\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.651339 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.651230    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-data\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.651682 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.651661    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istiod-ca-cert\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.653221 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.653193    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-podinfo\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.653345 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.653330    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-envoy\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.659805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.659750    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-token\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.659884 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.659871    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r8xf\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-kube-api-access-6r8xf\") pod \"e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") " pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:11.724195 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:11.724159    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:12.004903 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.004878    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"]
Sep 09 07:08:12.007296 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:08:12.007264    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba4368b_a777_4ef2_a28e_477f1bfa5aac.slice/crio-56c4df08c5043cfe542649ec0564c60b9671d0769597d220f1928c6144da1763 WatchSource:0}: Error finding container 56c4df08c5043cfe542649ec0564c60b9671d0769597d220f1928c6144da1763: Status 404 returned error can't find the container with id 56c4df08c5043cfe542649ec0564c60b9671d0769597d220f1928c6144da1763
Sep 09 07:08:12.009822 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.009787    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:12.009918 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.009874    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:12.009977 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.009915    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:12.059785 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.059757    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:12.249265 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.249218    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp" event={"ID":"4ba4368b-a777-4ef2-a28e-477f1bfa5aac","Type":"ContainerStarted","Data":"2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2"}
Sep 09 07:08:12.249423 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.249280    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp" event={"ID":"4ba4368b-a777-4ef2-a28e-477f1bfa5aac","Type":"ContainerStarted","Data":"56c4df08c5043cfe542649ec0564c60b9671d0769597d220f1928c6144da1763"}
Sep 09 07:08:12.271637 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.271518    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp" podStartSLOduration=1.271500772 podStartE2EDuration="1.271500772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:08:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:08:12.269906426 +0000 UTC m=+1024.640165515" watchObservedRunningTime="2026-09-09 07:08:12.271500772 +0000 UTC m=+1024.641759859"
Sep 09 07:08:12.542484 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.542403    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:12.642410 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.642376    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:12.724487 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.724456    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:12.750704 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:12.750665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:13.730457 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:13.730425    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:14.259824 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:14.259791    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:14.260711 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:14.260691    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:08:15.170467 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.170409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:15.181465 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.181430    2563 patch_prober.go:28] interesting pod/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 07:08:15.181627 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.181486    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 07:08:15.271571 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.271538    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:15.423036 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.423008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:15.889687 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.889663    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:08:15.983741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.983709    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:15.995810 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.995776    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663936c-b2d0-4e74-a46e-8b1417086440-istiod-ca-cert\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996032 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.995870    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-socket\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996032 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.995906    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-data\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996032 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.995937    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-credential-socket\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996032 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.995961    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663936c-b2d0-4e74-a46e-8b1417086440-istio-podinfo\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996032 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.995985    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-envoy\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996032 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.996018    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9dtbb\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-kube-api-access-9dtbb\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996397 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.996054    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-istio-token\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996397 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.996115    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-certs\") pod \"9663936c-b2d0-4e74-a46e-8b1417086440\" (UID: \"9663936c-b2d0-4e74-a46e-8b1417086440\") "
Sep 09 07:08:15.996397 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.996204    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9663936c-b2d0-4e74-a46e-8b1417086440-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:08:15.998538 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.996513    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9663936c-b2d0-4e74-a46e-8b1417086440-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:15.998538 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.996738    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:08:15.998538 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.996754    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-data" (OuterVolumeSpecName: "istio-data") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:08:15.998538 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.996940    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:08:15.998944 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.998915    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:08:15.999643 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.999340    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-kube-api-access-9dtbb" (OuterVolumeSpecName: "kube-api-access-9dtbb") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "kube-api-access-9dtbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:08:15.999643 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:15.999426    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:08:16.000958 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.000894    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-istio-token" (OuterVolumeSpecName: "istio-token") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:08:16.001405 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.001378    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9663936c-b2d0-4e74-a46e-8b1417086440-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9663936c-b2d0-4e74-a46e-8b1417086440" (UID: "9663936c-b2d0-4e74-a46e-8b1417086440"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:08:16.097220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.097170    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:16.097220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.097212    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:16.097220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.097225    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:16.097466 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.097239    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9663936c-b2d0-4e74-a46e-8b1417086440-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:16.097466 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.097251    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:16.097466 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.097262    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9dtbb\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-kube-api-access-9dtbb\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:16.097466 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.097274    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9663936c-b2d0-4e74-a46e-8b1417086440-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:16.097466 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.097284    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9663936c-b2d0-4e74-a46e-8b1417086440-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:16.271929 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.271834    2563 generic.go:356] "Generic (PLEG): container finished" podID="9663936c-b2d0-4e74-a46e-8b1417086440" containerID="0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8" exitCode=0
Sep 09 07:08:16.271929 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.271882    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"
Sep 09 07:08:16.272438 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.271924    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" event={"ID":"9663936c-b2d0-4e74-a46e-8b1417086440","Type":"ContainerDied","Data":"0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8"}
Sep 09 07:08:16.272438 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.271971    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk" event={"ID":"9663936c-b2d0-4e74-a46e-8b1417086440","Type":"ContainerDied","Data":"69e47abe5515e64c22e4e6072dc100f91b858ed710b913a1c74e4328cd7c1de6"}
Sep 09 07:08:16.272438 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.271994    2563 scope.go:117] "RemoveContainer" containerID="0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8"
Sep 09 07:08:16.281356 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.281338    2563 scope.go:117] "RemoveContainer" containerID="f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f"
Sep 09 07:08:16.289338 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.289322    2563 scope.go:117] "RemoveContainer" containerID="0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8"
Sep 09 07:08:16.289601 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:08:16.289571    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8\": container with ID starting with 0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8 not found: ID does not exist" containerID="0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8"
Sep 09 07:08:16.289678 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.289657    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8"} err="failed to get container status \"0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8\": rpc error: code = NotFound desc = could not find container \"0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8\": container with ID starting with 0015a264bbaaf0ae13a71c169427d4359b66dd0aee33fcc4aa911dd4649daba8 not found: ID does not exist"
Sep 09 07:08:16.289678 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.289675    2563 scope.go:117] "RemoveContainer" containerID="f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f"
Sep 09 07:08:16.289928 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:08:16.289912    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f\": container with ID starting with f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f not found: ID does not exist" containerID="f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f"
Sep 09 07:08:16.289970 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.289935    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f"} err="failed to get container status \"f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f\": rpc error: code = NotFound desc = could not find container \"f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f\": container with ID starting with f94e13914fb141cee703dfa895c78e5456b3edd0f7bfef2f473afaba63ae1f6f not found: ID does not exist"
Sep 09 07:08:16.300300 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.300274    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"]
Sep 09 07:08:16.306949 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:16.306929    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-a1a33e-openshift-default-6c54dfc6bd-zvtdk"]
Sep 09 07:08:17.222853 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:17.222806    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:18.021031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:18.021001    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:18.118101 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:18.118070    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" path="/var/lib/kubelet/pods/9663936c-b2d0-4e74-a46e-8b1417086440/volumes"
Sep 09 07:08:23.841886 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:23.841849    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:23.906620 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:23.906570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:23.994375 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:23.994336    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:24.094615 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:24.094512    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:24.194422 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:24.194387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:24.773017 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:24.772984    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:24.874698 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:24.874665    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:24.970872 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:24.970836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:26.248882 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.248847    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"]
Sep 09 07:08:26.249350 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.249076    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2" podUID="a5cc1a83-5420-4190-8e72-5eb49f523597" containerName="maas-api" containerID="cri-o://f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2" gracePeriod=30
Sep 09 07:08:26.282573 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.282535    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:26.382943 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.382913    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:26.485855 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.485823    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:26.508274 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.508223    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:08:26.584616 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.584561    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:26.602747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.602709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cc1a83-5420-4190-8e72-5eb49f523597-openshift-service-ca\") pod \"a5cc1a83-5420-4190-8e72-5eb49f523597\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") "
Sep 09 07:08:26.602852 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.602783    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls\") pod \"a5cc1a83-5420-4190-8e72-5eb49f523597\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") "
Sep 09 07:08:26.602911 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.602878    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-metrics-certs\") pod \"a5cc1a83-5420-4190-8e72-5eb49f523597\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") "
Sep 09 07:08:26.602974 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.602935    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2r4f4\" (UniqueName: \"kubernetes.io/projected/a5cc1a83-5420-4190-8e72-5eb49f523597-kube-api-access-2r4f4\") pod \"a5cc1a83-5420-4190-8e72-5eb49f523597\" (UID: \"a5cc1a83-5420-4190-8e72-5eb49f523597\") "
Sep 09 07:08:26.603133 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.603110    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5cc1a83-5420-4190-8e72-5eb49f523597-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a5cc1a83-5420-4190-8e72-5eb49f523597" (UID: "a5cc1a83-5420-4190-8e72-5eb49f523597"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:08:26.603294 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.603276    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a5cc1a83-5420-4190-8e72-5eb49f523597-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:26.605033 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.605004    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a5cc1a83-5420-4190-8e72-5eb49f523597" (UID: "a5cc1a83-5420-4190-8e72-5eb49f523597"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:08:26.605033 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.605019    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a5cc1a83-5420-4190-8e72-5eb49f523597" (UID: "a5cc1a83-5420-4190-8e72-5eb49f523597"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:08:26.605188 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.605129    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5cc1a83-5420-4190-8e72-5eb49f523597-kube-api-access-2r4f4" (OuterVolumeSpecName: "kube-api-access-2r4f4") pod "a5cc1a83-5420-4190-8e72-5eb49f523597" (UID: "a5cc1a83-5420-4190-8e72-5eb49f523597"). InnerVolumeSpecName "kube-api-access-2r4f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:08:26.704521 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.704489    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:26.704521 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.704516    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2r4f4\" (UniqueName: \"kubernetes.io/projected/a5cc1a83-5420-4190-8e72-5eb49f523597-kube-api-access-2r4f4\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:26.704521 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.704526    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a5cc1a83-5420-4190-8e72-5eb49f523597-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:08:26.816141 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:26.816056    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:27.330751 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.330713    2563 generic.go:356] "Generic (PLEG): container finished" podID="a5cc1a83-5420-4190-8e72-5eb49f523597" containerID="f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2" exitCode=0
Sep 09 07:08:27.330751 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.330756    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"
Sep 09 07:08:27.331252 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.330797    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2" event={"ID":"a5cc1a83-5420-4190-8e72-5eb49f523597","Type":"ContainerDied","Data":"f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2"}
Sep 09 07:08:27.331252 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.330848    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2" event={"ID":"a5cc1a83-5420-4190-8e72-5eb49f523597","Type":"ContainerDied","Data":"751ab00cb02e43a49e0a797cd41d27836ef18048e3c247345a2f103edf3d3eb4"}
Sep 09 07:08:27.331252 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.330870    2563 scope.go:117] "RemoveContainer" containerID="f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2"
Sep 09 07:08:27.340810 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.340790    2563 scope.go:117] "RemoveContainer" containerID="f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2"
Sep 09 07:08:27.341063 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:08:27.341047    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2\": container with ID starting with f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2 not found: ID does not exist" containerID="f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2"
Sep 09 07:08:27.341113 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.341070    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2"} err="failed to get container status \"f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2\": rpc error: code = NotFound desc = could not find container \"f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2\": container with ID starting with f91084dfbc7136a7938329b7de01697d0cd3516f14a70a5bdfeee9b267b6b7f2 not found: ID does not exist"
Sep 09 07:08:27.352755 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.352731    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"]
Sep 09 07:08:27.356774 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:27.356752    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fc9900-6dd47fc97d-9h7k2"]
Sep 09 07:08:28.117506 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:28.117474    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5cc1a83-5420-4190-8e72-5eb49f523597" path="/var/lib/kubelet/pods/a5cc1a83-5420-4190-8e72-5eb49f523597/volumes"
Sep 09 07:08:32.736196 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:32.736160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:32.882350 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:32.882316    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:33.441984 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:33.441949    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:33.547514 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:33.547473    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:33.609160 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:33.609131    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:33.806229 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:33.806139    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:34.361990 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.361955    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:34.637316 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637225    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"]
Sep 09 07:08:34.637711 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637693    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5cc1a83-5420-4190-8e72-5eb49f523597" containerName="maas-api"
Sep 09 07:08:34.637711 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637710    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5cc1a83-5420-4190-8e72-5eb49f523597" containerName="maas-api"
Sep 09 07:08:34.637890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637720    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy"
Sep 09 07:08:34.637890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637726    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy"
Sep 09 07:08:34.637890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637743    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy"
Sep 09 07:08:34.637890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637748    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy"
Sep 09 07:08:34.637890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637828    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy"
Sep 09 07:08:34.637890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637839    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9663936c-b2d0-4e74-a46e-8b1417086440" containerName="istio-proxy"
Sep 09 07:08:34.637890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.637881    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5cc1a83-5420-4190-8e72-5eb49f523597" containerName="maas-api"
Sep 09 07:08:34.641080 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.641059    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.644200 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.644180    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-ea0f1c-openshift-default-dockercfg-lb7nj\""
Sep 09 07:08:34.653441 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.653419    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"]
Sep 09 07:08:34.666095 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.666073    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:34.675772 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.675744    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-credential-socket\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.675880 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.675813    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-podinfo\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.675880 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.675857    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8s8tn\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-kube-api-access-8s8tn\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.675996 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.675929    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-socket\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.675996 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.675970    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-certs\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.676104 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.676058    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-envoy\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.676104 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.676087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb80169-0b6d-4d17-9a4d-d835558960f1-istiod-ca-cert\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.676208 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.676121    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-data\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.676208 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.676165    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-token\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777215 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777177    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-token\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777230    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-credential-socket\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777268    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-podinfo\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777284    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8s8tn\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-kube-api-access-8s8tn\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777301    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-socket\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777320    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-certs\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777767 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777388    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-envoy\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777767 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777417    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb80169-0b6d-4d17-9a4d-d835558960f1-istiod-ca-cert\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777767 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777633    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-data\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777946 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777782    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-credential-socket\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777946 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-socket\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.777946 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777890    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-certs\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.778093 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.777957    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-data\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.778093 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.778087    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb80169-0b6d-4d17-9a4d-d835558960f1-istiod-ca-cert\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.779743 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.779722    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-envoy\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.779838 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.779821    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-podinfo\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.786796 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.786768    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-token\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.786910 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.786895    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8s8tn\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-kube-api-access-8s8tn\") pod \"e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") " pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:34.952300 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:34.952271    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:35.228023 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.228000    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"]
Sep 09 07:08:35.230412 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:08:35.230384    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cb80169_0b6d_4d17_9a4d_d835558960f1.slice/crio-e638cb5844ae17bba442e7c74da617cdc0293ea59e386b1e4f36247518c5feb8 WatchSource:0}: Error finding container e638cb5844ae17bba442e7c74da617cdc0293ea59e386b1e4f36247518c5feb8: Status 404 returned error can't find the container with id e638cb5844ae17bba442e7c74da617cdc0293ea59e386b1e4f36247518c5feb8
Sep 09 07:08:35.233102 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.233065    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:35.233191 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.233133    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:35.233191 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.233160    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:35.374657 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.374618    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" event={"ID":"5cb80169-0b6d-4d17-9a4d-d835558960f1","Type":"ContainerStarted","Data":"0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37"}
Sep 09 07:08:35.374830 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.374680    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" event={"ID":"5cb80169-0b6d-4d17-9a4d-d835558960f1","Type":"ContainerStarted","Data":"e638cb5844ae17bba442e7c74da617cdc0293ea59e386b1e4f36247518c5feb8"}
Sep 09 07:08:35.407437 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.407371    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" podStartSLOduration=1.407349064 podStartE2EDuration="1.407349064s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:08:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:08:35.4051906 +0000 UTC m=+1047.775449702" watchObservedRunningTime="2026-09-09 07:08:35.407349064 +0000 UTC m=+1047.777608155"
Sep 09 07:08:35.428061 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.428028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:35.953240 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:35.953196    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:36.958248 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:36.958218    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:37.285456 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:37.285376    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:37.377524 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:37.377490    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:37.383946 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:37.383918    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:37.384772 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:37.384756    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:38.520254 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.520220    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"]
Sep 09 07:08:38.523487 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.523470    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.526351 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.526325    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-ea0f1c-serving-cert\""
Sep 09 07:08:38.537202 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.537177    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"]
Sep 09 07:08:38.573425 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.573387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:38.617505 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.617465    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.617714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.617512    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdqsx\" (UniqueName: \"kubernetes.io/projected/04d2e308-9a02-4a1b-aa34-b94adc09dacd-kube-api-access-mdqsx\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.617714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.617667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.617714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.617694    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d2e308-9a02-4a1b-aa34-b94adc09dacd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.683040 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.683008    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:38.719102 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.719073    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.719249 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.719109    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdqsx\" (UniqueName: \"kubernetes.io/projected/04d2e308-9a02-4a1b-aa34-b94adc09dacd-kube-api-access-mdqsx\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.719249 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.719168    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.719249 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.719184    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d2e308-9a02-4a1b-aa34-b94adc09dacd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.719423 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:08:38.719248    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-serving-cert: secret "maas-api-e2e-shared-a-w3-ea0f1c-serving-cert" not found
Sep 09 07:08:38.719423 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:08:38.719329    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls podName:04d2e308-9a02-4a1b-aa34-b94adc09dacd nodeName:}" failed. No retries permitted until 2026-09-09 07:08:39.219306254 +0000 UTC m=+1051.589565341 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls") pod "maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz" (UID: "04d2e308-9a02-4a1b-aa34-b94adc09dacd") : secret "maas-api-e2e-shared-a-w3-ea0f1c-serving-cert" not found
Sep 09 07:08:38.719872 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.719845    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d2e308-9a02-4a1b-aa34-b94adc09dacd-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.721577 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.721556    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.729214 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.729193    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdqsx\" (UniqueName: \"kubernetes.io/projected/04d2e308-9a02-4a1b-aa34-b94adc09dacd-kube-api-access-mdqsx\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:38.905131 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:38.905044    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:39.179246 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:39.179208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:39.224493 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:39.224460    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:39.226876 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:39.226849    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:39.280753 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:39.280716    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:39.437746 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:39.437668    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:39.711054 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:39.711027    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"]
Sep 09 07:08:39.712738 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:08:39.712708    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04d2e308_9a02_4a1b_aa34_b94adc09dacd.slice/crio-00cb612e7d02da73101a8ea5d5d7538abab821b4b68aaa0f025733d964e3924e WatchSource:0}: Error finding container 00cb612e7d02da73101a8ea5d5d7538abab821b4b68aaa0f025733d964e3924e: Status 404 returned error can't find the container with id 00cb612e7d02da73101a8ea5d5d7538abab821b4b68aaa0f025733d964e3924e
Sep 09 07:08:40.216757 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:40.216722    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:40.318746 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:40.318715    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:40.401892 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:40.401861    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz" event={"ID":"04d2e308-9a02-4a1b-aa34-b94adc09dacd","Type":"ContainerStarted","Data":"d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787"}
Sep 09 07:08:40.402071 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:40.401923    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz" event={"ID":"04d2e308-9a02-4a1b-aa34-b94adc09dacd","Type":"ContainerStarted","Data":"00cb612e7d02da73101a8ea5d5d7538abab821b4b68aaa0f025733d964e3924e"}
Sep 09 07:08:40.402071 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:40.401953    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:40.421877 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:40.421834    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz" podStartSLOduration=1.8520641439999999 podStartE2EDuration="2.421819181s" totalImagesPullingTime="569.755037ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:08:40.420857794 +0000 UTC m=+1052.791116882" watchObservedRunningTime="2026-09-09 07:08:40.421819181 +0000 UTC m=+1052.792078269"
Sep 09 07:08:40.481022 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:40.480926    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:40.761272 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:40.761167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:41.018957 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:41.018832    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:43.645521 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:43.645456    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:43.746095 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:43.746060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:44.861854 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:44.861815    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:44.982264 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:44.982214    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:45.268097 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:45.268067    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:45.490150 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:45.490118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:45.590088 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:45.590006    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:08:46.412983 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:46.412952    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:08:52.386148 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:52.386113    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 07:08:52.386612 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:52.386183    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 07:08:52.478109 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:52.478077    2563 generic.go:356] "Generic (PLEG): container finished" podID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerID="0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37" exitCode=0
Sep 09 07:08:52.478275 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:52.478151    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" event={"ID":"5cb80169-0b6d-4d17-9a4d-d835558960f1","Type":"ContainerDied","Data":"0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37"}
Sep 09 07:08:52.478552 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:52.478539    2563 scope.go:117] "RemoveContainer" containerID="0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37"
Sep 09 07:08:52.479646 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:52.479619    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:52.479715 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:52.479702    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:52.479762 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:52.479749    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:08:53.486328 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:53.486292    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" event={"ID":"5cb80169-0b6d-4d17-9a4d-d835558960f1","Type":"ContainerStarted","Data":"826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f"}
Sep 09 07:08:53.953162 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:53.953127    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:53.953449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:53.953171    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 07:08:53.953718 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:53.953688    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 07:08:54.953501 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:54.953457    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 07:08:54.954011 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:54.953574    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 07:08:55.953353 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:55.953302    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 07:08:55.953573 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:55.953379    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 07:08:56.952912 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:56.952874    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 07:08:56.953227 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:56.953198    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 07:08:57.953242 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:57.953193    2563 patch_prober.go:28] interesting pod/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body=
Sep 09 07:08:57.953768 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:57.953269    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused"
Sep 09 07:08:58.957191 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:58.957160    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:58.957554 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:58.957498    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:58.958136 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:58.958118    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:08:59.736527 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:08:59.736492    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:02.260606 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:02.260483    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:02.604456 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:02.604373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:02.967823 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:02.967783    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:03.208294 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:03.208260    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:05.701536 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:05.701484    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:05.794740 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:05.794700    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:06.010486 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:06.010397    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:07.315415 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.315382    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"]
Sep 09 07:09:07.319402 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.319379    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.322193 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.322169    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-988b24-serving-cert\""
Sep 09 07:09:07.333258 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.333232    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"]
Sep 09 07:09:07.354194 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.354167    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:07.417556 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.417528    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.417747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.417568    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ae422c-195e-4464-b7b8-2dec55397344-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.417747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.417671    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qm2h2\" (UniqueName: \"kubernetes.io/projected/42ae422c-195e-4464-b7b8-2dec55397344-kube-api-access-qm2h2\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.417747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.417740    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.459466 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.459430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:07.518379 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.518345    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.518561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.518416    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.518561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.518449    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ae422c-195e-4464-b7b8-2dec55397344-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.518561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.518491    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qm2h2\" (UniqueName: \"kubernetes.io/projected/42ae422c-195e-4464-b7b8-2dec55397344-kube-api-access-qm2h2\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.519168 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.519141    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ae422c-195e-4464-b7b8-2dec55397344-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.520820 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.520802    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.520878 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.520854    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.528118 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.528100    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qm2h2\" (UniqueName: \"kubernetes.io/projected/42ae422c-195e-4464-b7b8-2dec55397344-kube-api-access-qm2h2\") pod \"maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.559674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.559649    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:07.631689 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.631578    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:07.822893 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.822845    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:07.918496 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.918469    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"]
Sep 09 07:09:07.921075 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:09:07.921048    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42ae422c_195e_4464_b7b8_2dec55397344.slice/crio-741fe602e00ef167364a355f94521fcfa188f7757d7ca0b6e6b2dd115c6c0ba2 WatchSource:0}: Error finding container 741fe602e00ef167364a355f94521fcfa188f7757d7ca0b6e6b2dd115c6c0ba2: Status 404 returned error can't find the container with id 741fe602e00ef167364a355f94521fcfa188f7757d7ca0b6e6b2dd115c6c0ba2
Sep 09 07:09:07.936703 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:07.936661    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:08.260614 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:08.260572    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:08.567461 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:08.567345    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl" event={"ID":"42ae422c-195e-4464-b7b8-2dec55397344","Type":"ContainerStarted","Data":"51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e"}
Sep 09 07:09:08.567461 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:08.567417    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl" event={"ID":"42ae422c-195e-4464-b7b8-2dec55397344","Type":"ContainerStarted","Data":"741fe602e00ef167364a355f94521fcfa188f7757d7ca0b6e6b2dd115c6c0ba2"}
Sep 09 07:09:08.567901 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:08.567557    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:08.570287 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:08.570267    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:08.592524 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:08.592483    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl" podStartSLOduration=1.226580763 podStartE2EDuration="1.592469692s" totalImagesPullingTime="365.888929ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:09:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:09:08.589967666 +0000 UTC m=+1080.960226761" watchObservedRunningTime="2026-09-09 07:09:08.592469692 +0000 UTC m=+1080.962728779"
Sep 09 07:09:09.134917 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:09.134888    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:09.255121 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:09.255093    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:09.368709 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:09.368675    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:09.681618 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:09.681559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:14.578619 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:14.578575    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:09:15.244870 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:15.244838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:15.303278 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:15.303246    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:15.508664 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:15.505002    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:15.604655 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:15.604614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:15.957867 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:15.957816    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:18.578900 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:18.578866    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:20.720793 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:20.720764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:20.820718 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:20.820682    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"]
Sep 09 07:09:20.820914 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:20.820873    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp" podUID="4ba4368b-a777-4ef2-a28e-477f1bfa5aac" containerName="istio-proxy" containerID="cri-o://2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2" gracePeriod=30
Sep 09 07:09:20.847427 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:20.847311    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:21.075741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:21.075658    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:21.221832 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:21.221800    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:26.077726 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.077669    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:09:26.212274 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212245    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-token\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212438 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212321    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-data\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212503 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212483    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istiod-ca-cert\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212541    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-credential-socket\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212615 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212568    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-data" (OuterVolumeSpecName: "istio-data") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:09:26.212664 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212636    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-envoy\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212718 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212677    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-podinfo\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212777 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212756    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6r8xf\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-kube-api-access-6r8xf\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212841 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212799    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-socket\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212841 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212798    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:09:26.212944 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212867    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-certs\") pod \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\" (UID: \"4ba4368b-a777-4ef2-a28e-477f1bfa5aac\") "
Sep 09 07:09:26.212944 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.212883    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:09:26.213128 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.213076    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:09:26.213269 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.213209    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:09:26.213339 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.213287    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.213339 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.213302    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.213339 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.213315    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.213339 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.213328    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.213557 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.213341    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.214833 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.214810    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-token" (OuterVolumeSpecName: "istio-token") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:09:26.214947 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.214901    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:09:26.215192 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.215175    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:09:26.215268 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.215195    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-kube-api-access-6r8xf" (OuterVolumeSpecName: "kube-api-access-6r8xf") pod "4ba4368b-a777-4ef2-a28e-477f1bfa5aac" (UID: "4ba4368b-a777-4ef2-a28e-477f1bfa5aac"). InnerVolumeSpecName "kube-api-access-6r8xf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:09:26.227898 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.227874    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:26.314937 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.314905    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.314937 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.314934    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.314937 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.314944    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6r8xf\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-kube-api-access-6r8xf\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.315158 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.314953    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4ba4368b-a777-4ef2-a28e-477f1bfa5aac-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:09:26.330740 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.330666    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:26.428645 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.428611    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:26.531065 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.531032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:26.589317 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.589238    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:26.664070 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.664036    2563 generic.go:356] "Generic (PLEG): container finished" podID="4ba4368b-a777-4ef2-a28e-477f1bfa5aac" containerID="2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2" exitCode=0
Sep 09 07:09:26.664421 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.664087    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp" event={"ID":"4ba4368b-a777-4ef2-a28e-477f1bfa5aac","Type":"ContainerDied","Data":"2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2"}
Sep 09 07:09:26.664421 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.664090    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"
Sep 09 07:09:26.664421 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.664130    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp" event={"ID":"4ba4368b-a777-4ef2-a28e-477f1bfa5aac","Type":"ContainerDied","Data":"56c4df08c5043cfe542649ec0564c60b9671d0769597d220f1928c6144da1763"}
Sep 09 07:09:26.664421 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.664151    2563 scope.go:117] "RemoveContainer" containerID="2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2"
Sep 09 07:09:26.677696 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.677666    2563 scope.go:117] "RemoveContainer" containerID="2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2"
Sep 09 07:09:26.678026 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:09:26.678006    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2\": container with ID starting with 2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2 not found: ID does not exist" containerID="2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2"
Sep 09 07:09:26.678111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.678040    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2"} err="failed to get container status \"2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2\": rpc error: code = NotFound desc = could not find container \"2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2\": container with ID starting with 2a37e05099f1cf6057743091dc2f8e61140a75ef2366db02c76c34131e1a85f2 not found: ID does not exist"
Sep 09 07:09:26.693530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.693502    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"]
Sep 09 07:09:26.698094 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.698070    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-43ead3-openshift-default-5778fdc87b-zzvrp"]
Sep 09 07:09:26.894790 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:26.894711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:28.117600 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:28.117548    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ba4368b-a777-4ef2-a28e-477f1bfa5aac" path="/var/lib/kubelet/pods/4ba4368b-a777-4ef2-a28e-477f1bfa5aac/volumes"
Sep 09 07:09:28.328667 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:28.328633    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:30.699048 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:30.699012    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:32.705918 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:32.705881    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:32.806077 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:32.806041    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:32.980553 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:32.980470    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:33.317748 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:33.317645    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:33.416476 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:33.416441    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:33.559183 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:33.559149    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:34.250452 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:34.250409    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:39.824472 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:39.824433    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:39.938671 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:39.938637    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:40.024093 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:40.024054    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:40.190883 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:40.190848    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:46.043227 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:46.043193    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:46.143206 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:46.143177    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:46.994662 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:46.994626    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:47.366954 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:47.366869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:47.458910 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:47.458866    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:49.990163 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:49.990132    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:50.088557 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:50.088523    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:51.111454 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.111422    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"]
Sep 09 07:09:51.111919 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.111869    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ba4368b-a777-4ef2-a28e-477f1bfa5aac" containerName="istio-proxy"
Sep 09 07:09:51.111919 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.111882    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ba4368b-a777-4ef2-a28e-477f1bfa5aac" containerName="istio-proxy"
Sep 09 07:09:51.112005 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.111976    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ba4368b-a777-4ef2-a28e-477f1bfa5aac" containerName="istio-proxy"
Sep 09 07:09:51.115358 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.115335    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.118091 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.118069    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w2-1128ac-serving-cert\""
Sep 09 07:09:51.128115 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.128091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"]
Sep 09 07:09:51.152579 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.152548    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:51.160845 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.160810    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.160994 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.160878    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7c088fe-b76b-40b1-842e-7b664c7cd5db-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.161146 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.161118    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjgjz\" (UniqueName: \"kubernetes.io/projected/e7c088fe-b76b-40b1-842e-7b664c7cd5db-kube-api-access-cjgjz\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.161247 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.161168    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.253362 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.253328    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:51.262723 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.262699    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjgjz\" (UniqueName: \"kubernetes.io/projected/e7c088fe-b76b-40b1-842e-7b664c7cd5db-kube-api-access-cjgjz\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.262819 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.262732    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.262819 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.262771    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.262819 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.262814    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7c088fe-b76b-40b1-842e-7b664c7cd5db-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.262923 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:09:51.262882    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-serving-cert: secret "maas-api-e2e-shared-a-w2-1128ac-serving-cert" not found
Sep 09 07:09:51.262957 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:09:51.262942    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls podName:e7c088fe-b76b-40b1-842e-7b664c7cd5db nodeName:}" failed. No retries permitted until 2026-09-09 07:09:51.762926207 +0000 UTC m=+1124.133185277 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls") pod "maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6" (UID: "e7c088fe-b76b-40b1-842e-7b664c7cd5db") : secret "maas-api-e2e-shared-a-w2-1128ac-serving-cert" not found
Sep 09 07:09:51.263643 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.263618    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7c088fe-b76b-40b1-842e-7b664c7cd5db-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.265490 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.265465    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-metrics-certs\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.274135 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.274110    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjgjz\" (UniqueName: \"kubernetes.io/projected/e7c088fe-b76b-40b1-842e-7b664c7cd5db-kube-api-access-cjgjz\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.349203 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.349171    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:51.710196 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.710165    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:51.769030 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.768987    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.771316 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.771290    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls\") pod \"maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:51.813432 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:51.813402    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:09:52.027846 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:52.027763    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:52.307039 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:52.307014    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"]
Sep 09 07:09:52.309095 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:09:52.309068    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7c088fe_b76b_40b1_842e_7b664c7cd5db.slice/crio-b1248d66a3c40062fd4388ae45ff919f708b2f9b2ba6793a8f7197ac659a2a2d WatchSource:0}: Error finding container b1248d66a3c40062fd4388ae45ff919f708b2f9b2ba6793a8f7197ac659a2a2d: Status 404 returned error can't find the container with id b1248d66a3c40062fd4388ae45ff919f708b2f9b2ba6793a8f7197ac659a2a2d
Sep 09 07:09:52.803052 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:52.803014    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6" event={"ID":"e7c088fe-b76b-40b1-842e-7b664c7cd5db","Type":"ContainerStarted","Data":"b1248d66a3c40062fd4388ae45ff919f708b2f9b2ba6793a8f7197ac659a2a2d"}
Sep 09 07:09:53.809668 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:53.809626    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6" event={"ID":"e7c088fe-b76b-40b1-842e-7b664c7cd5db","Type":"ContainerStarted","Data":"27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1"}
Sep 09 07:09:53.810122 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:53.809736    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:09:53.832409 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:53.832347    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6" podStartSLOduration=2.330689486 podStartE2EDuration="2.832328189s" totalImagesPullingTime="501.638703ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:09:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:09:53.828068336 +0000 UTC m=+1126.198327421" watchObservedRunningTime="2026-09-09 07:09:53.832328189 +0000 UTC m=+1126.202587278"
Sep 09 07:09:59.820269 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:09:59.820237    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:10:03.382560 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:03.382525    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:03.481217 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:03.481182    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:11.386129 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:11.386064    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:12.999716 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:12.999680    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:13.100946 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:13.100883    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:15.025017 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:15.024970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:15.379542 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:15.379433    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:15.716843 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:15.716812    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:16.334484 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:16.334450    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:19.258935 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:19.258898    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:19.349880 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:19.349848    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:20.352615 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.352565    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"]
Sep 09 07:10:20.357692 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.357667    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.360438 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.360418    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-1d8843-serving-cert\""
Sep 09 07:10:20.365458 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.365421    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"]
Sep 09 07:10:20.389530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.389488    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:20.468985 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.468944    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.469159 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.469003    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e134e22-c8ff-455b-bee6-691f6d13e7fc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.469214 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.469110    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nk6k\" (UniqueName: \"kubernetes.io/projected/2e134e22-c8ff-455b-bee6-691f6d13e7fc-kube-api-access-8nk6k\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.469251 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.469222    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.497218 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.497181    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:20.570519 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.570483    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.570723 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.570524    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e134e22-c8ff-455b-bee6-691f6d13e7fc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.570723 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.570581    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nk6k\" (UniqueName: \"kubernetes.io/projected/2e134e22-c8ff-455b-bee6-691f6d13e7fc-kube-api-access-8nk6k\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.570723 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.570635    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.571249 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.571227    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e134e22-c8ff-455b-bee6-691f6d13e7fc-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.573000 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.572976    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.573097 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.573029    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.578535 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.578515    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nk6k\" (UniqueName: \"kubernetes.io/projected/2e134e22-c8ff-455b-bee6-691f6d13e7fc-kube-api-access-8nk6k\") pod \"maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.589673 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.589643    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:20.671123 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.671022    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:20.873196 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.873160    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:20.947826 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.947800    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"]
Sep 09 07:10:20.949785 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.949755    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:20.949928 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:10:20.949910    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e134e22_c8ff_455b_bee6_691f6d13e7fc.slice/crio-65161669a7d204b1c4c5e7dc87138ca6c9a7b6ddadfd7e9688f049f9e47f8cf6 WatchSource:0}: Error finding container 65161669a7d204b1c4c5e7dc87138ca6c9a7b6ddadfd7e9688f049f9e47f8cf6: Status 404 returned error can't find the container with id 65161669a7d204b1c4c5e7dc87138ca6c9a7b6ddadfd7e9688f049f9e47f8cf6
Sep 09 07:10:20.964221 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:20.964197    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb" event={"ID":"2e134e22-c8ff-455b-bee6-691f6d13e7fc","Type":"ContainerStarted","Data":"65161669a7d204b1c4c5e7dc87138ca6c9a7b6ddadfd7e9688f049f9e47f8cf6"}
Sep 09 07:10:21.971190 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:21.971149    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb" event={"ID":"2e134e22-c8ff-455b-bee6-691f6d13e7fc","Type":"ContainerStarted","Data":"8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13"}
Sep 09 07:10:21.971610 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:21.971223    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:21.991075 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:21.991023    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb" podStartSLOduration=1.477025103 podStartE2EDuration="1.991005417s" totalImagesPullingTime="513.980314ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:10:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:10:21.988553765 +0000 UTC m=+1154.358812853" watchObservedRunningTime="2026-09-09 07:10:21.991005417 +0000 UTC m=+1154.361264505"
Sep 09 07:10:27.982122 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:27.982082    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:10:44.237515 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:44.237424    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:57.305149 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.305115    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:57.589313 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.589239    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"]
Sep 09 07:10:57.593454 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.593429    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.596354 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.596332    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-0cc2c009-openshift-default-dockercfg-pknvj\""
Sep 09 07:10:57.605756 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.605730    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"]
Sep 09 07:10:57.621043 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.621018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:57.640568 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2b2xb\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-kube-api-access-2b2xb\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.640708 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640573    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-token\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.640708 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640624    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f17247-0b24-4dda-b672-cfd8a2f53e44-istiod-ca-cert\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.640708 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640700    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-credential-socket\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.640830 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640757    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-data\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.640830 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640794    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-envoy\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.640830 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640822    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-socket\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.640977 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640840    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-certs\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.640977 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.640896    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-podinfo\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.720870 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.720841    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:10:57.741796 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.741762    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-envoy\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.741963 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.741811    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-socket\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.741963 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.741831    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-certs\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.741963 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.741850    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-podinfo\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.741963 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.741917    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2b2xb\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-kube-api-access-2b2xb\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.741963 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.741943    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-token\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.741963 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.741960    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f17247-0b24-4dda-b672-cfd8a2f53e44-istiod-ca-cert\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.742294 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.741993    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-credential-socket\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.742294 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.742041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-data\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.742294 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.742179    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-certs\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.742294 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.742247    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-socket\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.742510 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.742331    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-data\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.742642 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.742609    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-credential-socket\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.742891 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.742869    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f17247-0b24-4dda-b672-cfd8a2f53e44-istiod-ca-cert\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.744247 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.744210    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-envoy\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.744530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.744511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-podinfo\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.751059 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.751026    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-token\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.751159 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.751036    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2b2xb\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-kube-api-access-2b2xb\") pod \"e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") " pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:57.904869 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:57.904777    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:58.183366 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:58.183340    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"]
Sep 09 07:10:58.185221 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:10:58.185194    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65f17247_0b24_4dda_b672_cfd8a2f53e44.slice/crio-dac20a38fc806d2b3769a4dd071e206a5152b98c24be2510e995231d573de268 WatchSource:0}: Error finding container dac20a38fc806d2b3769a4dd071e206a5152b98c24be2510e995231d573de268: Status 404 returned error can't find the container with id dac20a38fc806d2b3769a4dd071e206a5152b98c24be2510e995231d573de268
Sep 09 07:10:58.187894 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:58.187864    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:10:58.187972 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:58.187928    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:10:58.187972 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:58.187955    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:10:59.185737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:59.185697    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" event={"ID":"65f17247-0b24-4dda-b672-cfd8a2f53e44","Type":"ContainerStarted","Data":"a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1"}
Sep 09 07:10:59.186153 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:59.185782    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" event={"ID":"65f17247-0b24-4dda-b672-cfd8a2f53e44","Type":"ContainerStarted","Data":"dac20a38fc806d2b3769a4dd071e206a5152b98c24be2510e995231d573de268"}
Sep 09 07:10:59.208213 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:59.208163    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" podStartSLOduration=2.208145173 podStartE2EDuration="2.208145173s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:10:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:10:59.207426194 +0000 UTC m=+1191.577685284" watchObservedRunningTime="2026-09-09 07:10:59.208145173 +0000 UTC m=+1191.578404261"
Sep 09 07:10:59.905774 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:59.905737    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:10:59.911382 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:10:59.911357    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:11:00.191460 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:00.191426    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:11:00.192643 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:00.192623    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:11:01.098135 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:01.098098    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:01.196950 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:01.196916    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:02.326874 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.326838    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"]
Sep 09 07:11:02.331070 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.331045    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.333960 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.333934    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-0cc2c009-serving-cert\""
Sep 09 07:11:02.342386 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.342359    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"]
Sep 09 07:11:02.355346 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.355317    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:02.456989 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.456954    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:02.494342 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.494293    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.494342 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.494349    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc6v9\" (UniqueName: \"kubernetes.io/projected/15fedfe9-1094-443b-9f26-04ca1efb1c3e-kube-api-access-rc6v9\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.494546 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.494444    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-metrics-certs\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.494546 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.494514    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fedfe9-1094-443b-9f26-04ca1efb1c3e-openshift-service-ca\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.595263 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.595147    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-metrics-certs\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.595451 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.595273    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fedfe9-1094-443b-9f26-04ca1efb1c3e-openshift-service-ca\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.595451 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.595336    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.595451 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.595364    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc6v9\" (UniqueName: \"kubernetes.io/projected/15fedfe9-1094-443b-9f26-04ca1efb1c3e-kube-api-access-rc6v9\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.595652 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:11:02.595485    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-serving-cert: secret "maas-api-e2e-mt-0cc2c009-serving-cert" not found
Sep 09 07:11:02.595652 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:11:02.595556    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls podName:15fedfe9-1094-443b-9f26-04ca1efb1c3e nodeName:}" failed. No retries permitted until 2026-09-09 07:11:03.095539104 +0000 UTC m=+1195.465798188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls") pod "maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d" (UID: "15fedfe9-1094-443b-9f26-04ca1efb1c3e") : secret "maas-api-e2e-mt-0cc2c009-serving-cert" not found
Sep 09 07:11:02.596086 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.596068    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fedfe9-1094-443b-9f26-04ca1efb1c3e-openshift-service-ca\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.597672 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.597651    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-metrics-certs\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.603992 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.603969    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc6v9\" (UniqueName: \"kubernetes.io/projected/15fedfe9-1094-443b-9f26-04ca1efb1c3e-kube-api-access-rc6v9\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:02.698741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.698706    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:02.853879 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.853793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:02.955027 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:02.954969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:03.101076 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:03.101038    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:03.103670 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:03.103645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls\") pod \"maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:03.244327 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:03.244291    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:03.533124 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:03.533089    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"]
Sep 09 07:11:03.534975 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:11:03.534948    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15fedfe9_1094_443b_9f26_04ca1efb1c3e.slice/crio-61866295c82ba0afe4ae0e286c6b3e82a2f76fa758c313a4d2d279c95e089152 WatchSource:0}: Error finding container 61866295c82ba0afe4ae0e286c6b3e82a2f76fa758c313a4d2d279c95e089152: Status 404 returned error can't find the container with id 61866295c82ba0afe4ae0e286c6b3e82a2f76fa758c313a4d2d279c95e089152
Sep 09 07:11:04.215112 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:04.215032    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d" event={"ID":"15fedfe9-1094-443b-9f26-04ca1efb1c3e","Type":"ContainerStarted","Data":"b1003fe27d98aba2629f48d43263461b08dea513b37bc29c58e6e1a6b12575d2"}
Sep 09 07:11:04.215112 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:04.215081    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d" event={"ID":"15fedfe9-1094-443b-9f26-04ca1efb1c3e","Type":"ContainerStarted","Data":"61866295c82ba0afe4ae0e286c6b3e82a2f76fa758c313a4d2d279c95e089152"}
Sep 09 07:11:04.215330 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:04.215116    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:04.251818 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:04.251773    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d" podStartSLOduration=1.832211372 podStartE2EDuration="2.25175737s" totalImagesPullingTime="419.545998ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:11:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:11:04.25143806 +0000 UTC m=+1196.621697149" watchObservedRunningTime="2026-09-09 07:11:04.25175737 +0000 UTC m=+1196.622016457"
Sep 09 07:11:08.176969 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:08.176940    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:11:08.178694 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:08.178673    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:11:10.226043 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:10.226011    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:11:16.970269 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:16.970220    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:23.323941 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:23.323904    2563 generic.go:356] "Generic (PLEG): container finished" podID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerID="a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1" exitCode=0
Sep 09 07:11:23.324413 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:23.323980    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" event={"ID":"65f17247-0b24-4dda-b672-cfd8a2f53e44","Type":"ContainerDied","Data":"a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1"}
Sep 09 07:11:23.324487 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:23.324469    2563 scope.go:117] "RemoveContainer" containerID="a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1"
Sep 09 07:11:23.325656 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:23.325610    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:11:23.325728 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:23.325715    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:11:23.325775 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:23.325763    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:11:23.384057 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:23.384026    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:24.331527 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:24.331490    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" event={"ID":"65f17247-0b24-4dda-b672-cfd8a2f53e44","Type":"ContainerStarted","Data":"1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce"}
Sep 09 07:11:24.905785 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:24.905746    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:11:24.905971 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:24.905932    2563 patch_prober.go:28] interesting pod/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 07:11:24.906030 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:24.905987    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 07:11:25.906122 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:25.906087    2563 patch_prober.go:28] interesting pod/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 07:11:25.906505 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:25.906139    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 07:11:26.111514 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:26.111428    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:26.905306 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:26.905269    2563 patch_prober.go:28] interesting pod/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 07:11:26.905496 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:26.905323    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 07:11:27.905852 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:27.905817    2563 patch_prober.go:28] interesting pod/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 07:11:27.906256 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:27.905884    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 07:11:28.906119 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:28.906079    2563 patch_prober.go:28] interesting pod/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused" start-of-body=
Sep 09 07:11:28.906570 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:28.906153    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.62:15021/healthz/ready\": dial tcp 10.133.0.62:15021: connect: connection refused"
Sep 09 07:11:29.909557 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:29.909520    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:11:29.909940 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:29.909782    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:11:29.910406 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:29.910385    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:11:31.630079 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:31.630043    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:11:33.890146 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:11:33.890111    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:16.382753 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.382658    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"]
Sep 09 07:12:16.383620 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.382928    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d" podUID="15fedfe9-1094-443b-9f26-04ca1efb1c3e" containerName="maas-api" containerID="cri-o://b1003fe27d98aba2629f48d43263461b08dea513b37bc29c58e6e1a6b12575d2" gracePeriod=30
Sep 09 07:12:16.416784 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.416756    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:16.516407 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.516368    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:16.613755 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.613721    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:16.618771 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.618744    2563 generic.go:356] "Generic (PLEG): container finished" podID="15fedfe9-1094-443b-9f26-04ca1efb1c3e" containerID="b1003fe27d98aba2629f48d43263461b08dea513b37bc29c58e6e1a6b12575d2" exitCode=0
Sep 09 07:12:16.618900 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.618812    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d" event={"ID":"15fedfe9-1094-443b-9f26-04ca1efb1c3e","Type":"ContainerDied","Data":"b1003fe27d98aba2629f48d43263461b08dea513b37bc29c58e6e1a6b12575d2"}
Sep 09 07:12:16.637628 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.637530    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:12:16.718582 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.718540    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:16.754367 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.754337    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls\") pod \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") "
Sep 09 07:12:16.754536 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.754400    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-metrics-certs\") pod \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") "
Sep 09 07:12:16.754536 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.754437    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rc6v9\" (UniqueName: \"kubernetes.io/projected/15fedfe9-1094-443b-9f26-04ca1efb1c3e-kube-api-access-rc6v9\") pod \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") "
Sep 09 07:12:16.754536 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.754473    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fedfe9-1094-443b-9f26-04ca1efb1c3e-openshift-service-ca\") pod \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\" (UID: \"15fedfe9-1094-443b-9f26-04ca1efb1c3e\") "
Sep 09 07:12:16.755038 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.754990    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/15fedfe9-1094-443b-9f26-04ca1efb1c3e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "15fedfe9-1094-443b-9f26-04ca1efb1c3e" (UID: "15fedfe9-1094-443b-9f26-04ca1efb1c3e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:12:16.756701 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.756649    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "15fedfe9-1094-443b-9f26-04ca1efb1c3e" (UID: "15fedfe9-1094-443b-9f26-04ca1efb1c3e"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:12:16.756701 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.756670    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "15fedfe9-1094-443b-9f26-04ca1efb1c3e" (UID: "15fedfe9-1094-443b-9f26-04ca1efb1c3e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:12:16.756848 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.756770    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/15fedfe9-1094-443b-9f26-04ca1efb1c3e-kube-api-access-rc6v9" (OuterVolumeSpecName: "kube-api-access-rc6v9") pod "15fedfe9-1094-443b-9f26-04ca1efb1c3e" (UID: "15fedfe9-1094-443b-9f26-04ca1efb1c3e"). InnerVolumeSpecName "kube-api-access-rc6v9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:12:16.856363 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.856308    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:12:16.856363 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.856353    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/15fedfe9-1094-443b-9f26-04ca1efb1c3e-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:12:16.856363 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.856365    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc6v9\" (UniqueName: \"kubernetes.io/projected/15fedfe9-1094-443b-9f26-04ca1efb1c3e-kube-api-access-rc6v9\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:12:16.856363 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:16.856374    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/15fedfe9-1094-443b-9f26-04ca1efb1c3e-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:12:17.625582 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:17.625550    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"
Sep 09 07:12:17.626071 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:17.625578    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d" event={"ID":"15fedfe9-1094-443b-9f26-04ca1efb1c3e","Type":"ContainerDied","Data":"61866295c82ba0afe4ae0e286c6b3e82a2f76fa758c313a4d2d279c95e089152"}
Sep 09 07:12:17.626071 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:17.625663    2563 scope.go:117] "RemoveContainer" containerID="b1003fe27d98aba2629f48d43263461b08dea513b37bc29c58e6e1a6b12575d2"
Sep 09 07:12:17.650165 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:17.650140    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"]
Sep 09 07:12:17.655454 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:17.655429    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-0cc2c009-75db8b89d8-9k97d"]
Sep 09 07:12:18.117531 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:18.117496    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="15fedfe9-1094-443b-9f26-04ca1efb1c3e" path="/var/lib/kubelet/pods/15fedfe9-1094-443b-9f26-04ca1efb1c3e/volumes"
Sep 09 07:12:19.823730 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:19.823701    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:19.924238 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:19.924205    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:20.269187 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:20.269133    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:20.394413 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:20.394383    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:21.458270 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:21.458234    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:25.350365 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:25.350331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:27.035955 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:27.035922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:53.965147 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:53.965114    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:12:54.064336 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:12:54.064299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:02.102509 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:02.102471    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:02.463666 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:02.463634    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:08.246844 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:08.246805    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:08.347116 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:08.347080    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:08.533959 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:08.533871    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"]
Sep 09 07:13:08.534124 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:08.534079    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy" containerID="cri-o://1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce" gracePeriod=30
Sep 09 07:13:08.563312 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:08.563281    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:09.979197 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:09.979154    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:10.077939 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:10.077906    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:10.178149 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:10.178082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:11.038939 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:11.038908    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:11.133089 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:11.133060    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:11.233925 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:11.233885    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:12.643862 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:12.643827    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:13.793790 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.793730    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:13:13.820322 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.820286    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-credential-socket\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.820498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.820333    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-data\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.820498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.820373    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-podinfo\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.820498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.820397    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-token\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.820498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.820435    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f17247-0b24-4dda-b672-cfd8a2f53e44-istiod-ca-cert\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.820498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.820461    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-certs\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.820790 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.820528    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-envoy\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.821004 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.820898    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:13:13.821250 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.821195    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-data" (OuterVolumeSpecName: "istio-data") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:13:13.821334 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.821262    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/65f17247-0b24-4dda-b672-cfd8a2f53e44-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:13:13.821402 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.821377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-socket\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.821610 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.821471    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2b2xb\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-kube-api-access-2b2xb\") pod \"65f17247-0b24-4dda-b672-cfd8a2f53e44\" (UID: \"65f17247-0b24-4dda-b672-cfd8a2f53e44\") "
Sep 09 07:13:13.821852 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.821825    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:13:13.822051 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.822026    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:13:13.822235 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.822043    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.822352 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.822338    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.822453 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.822441    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/65f17247-0b24-4dda-b672-cfd8a2f53e44-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.822543 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.822531    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.823392 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.823360    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:13:13.824190 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.824168    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-token" (OuterVolumeSpecName: "istio-token") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:13:13.824272 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.824217    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-kube-api-access-2b2xb" (OuterVolumeSpecName: "kube-api-access-2b2xb") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "kube-api-access-2b2xb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:13:13.824340 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.824321    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "65f17247-0b24-4dda-b672-cfd8a2f53e44" (UID: "65f17247-0b24-4dda-b672-cfd8a2f53e44"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:13:13.923363 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.923325    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.923363 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.923357    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.923363 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.923371    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.923674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.923382    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/65f17247-0b24-4dda-b672-cfd8a2f53e44-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.923674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.923394    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2b2xb\" (UniqueName: \"kubernetes.io/projected/65f17247-0b24-4dda-b672-cfd8a2f53e44-kube-api-access-2b2xb\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:13:13.933344 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.933310    2563 generic.go:356] "Generic (PLEG): container finished" podID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerID="1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce" exitCode=0
Sep 09 07:13:13.933494 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.933364    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"
Sep 09 07:13:13.933494 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.933394    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" event={"ID":"65f17247-0b24-4dda-b672-cfd8a2f53e44","Type":"ContainerDied","Data":"1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce"}
Sep 09 07:13:13.933494 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.933453    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk" event={"ID":"65f17247-0b24-4dda-b672-cfd8a2f53e44","Type":"ContainerDied","Data":"dac20a38fc806d2b3769a4dd071e206a5152b98c24be2510e995231d573de268"}
Sep 09 07:13:13.933494 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.933477    2563 scope.go:117] "RemoveContainer" containerID="1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce"
Sep 09 07:13:13.943997 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.943980    2563 scope.go:117] "RemoveContainer" containerID="a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1"
Sep 09 07:13:13.952006 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.951989    2563 scope.go:117] "RemoveContainer" containerID="1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce"
Sep 09 07:13:13.952247 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:13:13.952228    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce\": container with ID starting with 1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce not found: ID does not exist" containerID="1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce"
Sep 09 07:13:13.952299 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.952255    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce"} err="failed to get container status \"1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce\": rpc error: code = NotFound desc = could not find container \"1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce\": container with ID starting with 1a5c5f3d9070330d2dc3ea6fa4c31126e268a31e12132155f56d16f99c64c0ce not found: ID does not exist"
Sep 09 07:13:13.952299 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.952271    2563 scope.go:117] "RemoveContainer" containerID="a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1"
Sep 09 07:13:13.952519 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:13:13.952504    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1\": container with ID starting with a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1 not found: ID does not exist" containerID="a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1"
Sep 09 07:13:13.952564 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.952523    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1"} err="failed to get container status \"a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1\": rpc error: code = NotFound desc = could not find container \"a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1\": container with ID starting with a57c1b34d214f2ee8ed003a04c21c5b62a9c7a57b0fdca670ec32131ef4771c1 not found: ID does not exist"
Sep 09 07:13:13.955698 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.955672    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"]
Sep 09 07:13:13.960776 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:13.960754    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-0cc2c009-openshift-default-58d4d98996-gwkrk"]
Sep 09 07:13:14.117865 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:14.117786    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" path="/var/lib/kubelet/pods/65f17247-0b24-4dda-b672-cfd8a2f53e44/volumes"
Sep 09 07:13:19.009793 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:19.009754    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:19.110743 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:19.110711    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:19.772781 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:19.772746    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:26.008373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:26.008331    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:26.206675 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:26.206642    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:26.438250 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:26.438216    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:36.821702 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:36.821614    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:37.064660 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:37.064623    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:39.516063 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:39.516025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:40.385653 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:40.385610    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:55.370325 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:55.370287    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:13:58.179510 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:13:58.179480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:00.656971 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:00.656934    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:04.126180 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:04.126150    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:04.191560 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:04.191527    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:04.553969 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:04.553938    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:05.725562 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:05.725526    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:05.787385 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:05.787341    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:07.744093 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:07.744059    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:07.844532 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:07.844501    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:08.956835 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.956800    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"]
Sep 09 07:14:08.957301 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.957257    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy"
Sep 09 07:14:08.957301 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.957270    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy"
Sep 09 07:14:08.957301 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.957300    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="15fedfe9-1094-443b-9f26-04ca1efb1c3e" containerName="maas-api"
Sep 09 07:14:08.957419 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.957310    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="15fedfe9-1094-443b-9f26-04ca1efb1c3e" containerName="maas-api"
Sep 09 07:14:08.957419 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.957386    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy"
Sep 09 07:14:08.957419 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.957397    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="15fedfe9-1094-443b-9f26-04ca1efb1c3e" containerName="maas-api"
Sep 09 07:14:08.961829 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.961811    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:08.965840 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.965820    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-043f83-serving-cert\""
Sep 09 07:14:08.976736 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:08.976712    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"]
Sep 09 07:14:09.019955 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.019913    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:09.048389 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.048352    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.048561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.048414    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nmp7\" (UniqueName: \"kubernetes.io/projected/4c8db05a-7cff-467b-9efd-add2e56a8b87-kube-api-access-6nmp7\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.048561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.048475    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8db05a-7cff-467b-9efd-add2e56a8b87-openshift-service-ca\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.048561 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.048516    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-metrics-certs\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.123398 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.123365    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:09.149306 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.149280    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-metrics-certs\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.149479 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.149347    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.149479 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.149392    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nmp7\" (UniqueName: \"kubernetes.io/projected/4c8db05a-7cff-467b-9efd-add2e56a8b87-kube-api-access-6nmp7\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.149479 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.149426    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8db05a-7cff-467b-9efd-add2e56a8b87-openshift-service-ca\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.149680 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:14:09.149511    2563 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-serving-cert: secret "maas-api-e2e-api-a-043f83-serving-cert" not found
Sep 09 07:14:09.149680 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:14:09.149622    2563 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls podName:4c8db05a-7cff-467b-9efd-add2e56a8b87 nodeName:}" failed. No retries permitted until 2026-09-09 07:14:09.649569178 +0000 UTC m=+1382.019828248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls") pod "maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv" (UID: "4c8db05a-7cff-467b-9efd-add2e56a8b87") : secret "maas-api-e2e-api-a-043f83-serving-cert" not found
Sep 09 07:14:09.150143 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.150122    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8db05a-7cff-467b-9efd-add2e56a8b87-openshift-service-ca\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.151879 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.151857    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-metrics-certs\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.158573 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.158551    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nmp7\" (UniqueName: \"kubernetes.io/projected/4c8db05a-7cff-467b-9efd-add2e56a8b87-kube-api-access-6nmp7\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.227706 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.227575    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:09.531061 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.530970    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:09.627901 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.627871    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:09.655087 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.655056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.657613 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.657569    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls\") pod \"maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:09.871860 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:09.871775    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:10.144166 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:10.144141    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"]
Sep 09 07:14:10.146337 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:14:10.146307    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c8db05a_7cff_467b_9efd_add2e56a8b87.slice/crio-3d9adc9650519ea44f1b82c279d58e6960b8e19c6587a3a786bbcf7533535f92 WatchSource:0}: Error finding container 3d9adc9650519ea44f1b82c279d58e6960b8e19c6587a3a786bbcf7533535f92: Status 404 returned error can't find the container with id 3d9adc9650519ea44f1b82c279d58e6960b8e19c6587a3a786bbcf7533535f92
Sep 09 07:14:10.147658 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:10.147643    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 07:14:10.234913 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:10.234879    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv" event={"ID":"4c8db05a-7cff-467b-9efd-add2e56a8b87","Type":"ContainerStarted","Data":"3d9adc9650519ea44f1b82c279d58e6960b8e19c6587a3a786bbcf7533535f92"}
Sep 09 07:14:11.241941 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:11.241902    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv" event={"ID":"4c8db05a-7cff-467b-9efd-add2e56a8b87","Type":"ContainerStarted","Data":"00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5"}
Sep 09 07:14:11.242360 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:11.242020    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:11.261531 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:11.261485    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv" podStartSLOduration=2.782272475 podStartE2EDuration="3.261472096s" totalImagesPullingTime="479.199621ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:14:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:14:11.259206492 +0000 UTC m=+1383.629465581" watchObservedRunningTime="2026-09-09 07:14:11.261472096 +0000 UTC m=+1383.631731183"
Sep 09 07:14:17.253073 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:17.253041    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:14:17.470127 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:17.470094    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:21.144575 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:21.144538    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:29.124225 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:29.124187    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:29.919137 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:29.919105    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:32.919513 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:32.919480    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:33.521148 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.521118    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:33.874655 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.874563    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"]
Sep 09 07:14:33.875045 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.875032    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy"
Sep 09 07:14:33.875094 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.875047    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy"
Sep 09 07:14:33.875137 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.875112    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="65f17247-0b24-4dda-b672-cfd8a2f53e44" containerName="istio-proxy"
Sep 09 07:14:33.878654 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.878631    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.885699 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.885677    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-d8398c-openshift-default-dockercfg-rttvs\""
Sep 09 07:14:33.896776 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.896753    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"]
Sep 09 07:14:33.912045 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.912012    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:33.996512 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.996473    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40df5d19-ec78-4f40-a782-a7afad941d53-istiod-ca-cert\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.997163 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.997087    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-data\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.997332 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.997317    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40df5d19-ec78-4f40-a782-a7afad941d53-istio-podinfo\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.997498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.997468    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-socket\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.997573 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.997551    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-envoy\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.997702 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.997674    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-credential-socket\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.997779 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.997720    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-istio-token\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.997842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.997778    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-certs\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:33.997842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:33.997812    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gr69\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-kube-api-access-4gr69\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.098634 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098602    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40df5d19-ec78-4f40-a782-a7afad941d53-istiod-ca-cert\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.098805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098656    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-data\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.098805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098684    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40df5d19-ec78-4f40-a782-a7afad941d53-istio-podinfo\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.098805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098714    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-socket\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.098805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098770    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-envoy\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099020 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098822    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-credential-socket\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099020 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098853    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-istio-token\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099020 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098896    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-certs\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099020 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.098927    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gr69\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-kube-api-access-4gr69\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099216 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.099079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-data\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099216 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.099138    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-socket\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099414 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.099391    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-certs\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099474 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.099456    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-credential-socket\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.099652 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.099518    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40df5d19-ec78-4f40-a782-a7afad941d53-istiod-ca-cert\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.101275 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.101250    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-envoy\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.101408 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.101390    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40df5d19-ec78-4f40-a782-a7afad941d53-istio-podinfo\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.111756 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.111728    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-istio-token\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.112051 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.111829    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gr69\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-kube-api-access-4gr69\") pod \"e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") " pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.189019 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.188994    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:34.465745 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.465719    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"]
Sep 09 07:14:34.467337 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:14:34.467310    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40df5d19_ec78_4f40_a782_a7afad941d53.slice/crio-13734a89b294ab480731b3e9b3ccd988ed11dd6b7360c4ffec1b9f95f4c60e5d WatchSource:0}: Error finding container 13734a89b294ab480731b3e9b3ccd988ed11dd6b7360c4ffec1b9f95f4c60e5d: Status 404 returned error can't find the container with id 13734a89b294ab480731b3e9b3ccd988ed11dd6b7360c4ffec1b9f95f4c60e5d
Sep 09 07:14:34.469656 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.469626    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:14:34.469714 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.469690    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:14:34.469751 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.469716    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:14:34.671614 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:34.671560    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:35.367305 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:35.367270    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" event={"ID":"40df5d19-ec78-4f40-a782-a7afad941d53","Type":"ContainerStarted","Data":"80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860"}
Sep 09 07:14:35.367748 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:35.367319    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" event={"ID":"40df5d19-ec78-4f40-a782-a7afad941d53","Type":"ContainerStarted","Data":"13734a89b294ab480731b3e9b3ccd988ed11dd6b7360c4ffec1b9f95f4c60e5d"}
Sep 09 07:14:35.389487 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:35.389445    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" podStartSLOduration=2.38943038 podStartE2EDuration="2.38943038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:14:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:14:35.38743391 +0000 UTC m=+1407.757692998" watchObservedRunningTime="2026-09-09 07:14:35.38943038 +0000 UTC m=+1407.759689467"
Sep 09 07:14:36.189309 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:36.189273    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:36.194454 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:36.194430    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:36.373350 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:36.373321    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:36.374260 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:36.374242    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:14:37.579504 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:37.579474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:37.679866 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:37.679826    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:38.752391 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:38.752361    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:38.852120 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:38.852085    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:39.291759 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:39.291718    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:39.506246 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:39.506208    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:39.580936 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:39.580843    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:14:58.503384 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:58.503350    2563 generic.go:356] "Generic (PLEG): container finished" podID="40df5d19-ec78-4f40-a782-a7afad941d53" containerID="80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860" exitCode=0
Sep 09 07:14:58.503824 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:58.503424    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" event={"ID":"40df5d19-ec78-4f40-a782-a7afad941d53","Type":"ContainerDied","Data":"80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860"}
Sep 09 07:14:58.503884 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:58.503868    2563 scope.go:117] "RemoveContainer" containerID="80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860"
Sep 09 07:14:58.505047 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:58.505018    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:14:58.505106 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:58.505085    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:14:58.505142 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:58.505118    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:14:59.511983 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:14:59.511945    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" event={"ID":"40df5d19-ec78-4f40-a782-a7afad941d53","Type":"ContainerStarted","Data":"867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82"}
Sep 09 07:15:00.190133 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:00.190085    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:15:00.190353 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:00.190327    2563 patch_prober.go:28] interesting pod/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 07:15:00.190465 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:00.190437    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 07:15:01.189963 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:01.189925    2563 patch_prober.go:28] interesting pod/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 07:15:01.190375 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:01.189991    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 07:15:02.190058 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:02.190020    2563 patch_prober.go:28] interesting pod/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 07:15:02.190546 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:02.190088    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 07:15:02.983435 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:02.983405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:03.189683 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:03.189645    2563 patch_prober.go:28] interesting pod/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 07:15:03.189858 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:03.189716    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 07:15:04.189841 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:04.189805    2563 patch_prober.go:28] interesting pod/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 07:15:04.190234 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:04.189858    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 07:15:05.189480 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:05.189442    2563 patch_prober.go:28] interesting pod/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body=
Sep 09 07:15:05.189678 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:05.189497    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused"
Sep 09 07:15:06.194533 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:06.194500    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:15:06.195151 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:06.194794    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:15:06.195468 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:06.195452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:15:06.858442 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:06.858408    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:08.786957 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:08.786922    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:09.731048 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:09.731018    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:10.354987 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.354956    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:10.711015 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.710980    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"]
Sep 09 07:15:10.723536 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.723498    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.726731 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.726703    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-bea943-openshift-default-dockercfg-c2hpl\""
Sep 09 07:15:10.730224 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.730191    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"]
Sep 09 07:15:10.746844 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.746814    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:10.784136 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784105    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-token\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.784303 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784164    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-envoy\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.784303 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784250    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-socket\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.784428 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784311    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-credential-socket\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.784428 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784359    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-podinfo\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.784428 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784383    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-data\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.784616 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784446    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-certs\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.784616 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784483    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ba2879-52db-4cc4-b25f-123c48f9556c-istiod-ca-cert\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.784616 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.784566    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5sdrc\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-kube-api-access-5sdrc\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886043 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886004    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-data\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886062    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-certs\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886090    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ba2879-52db-4cc4-b25f-123c48f9556c-istiod-ca-cert\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886146    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5sdrc\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-kube-api-access-5sdrc\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886371 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886251    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-token\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886371 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886288    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-envoy\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886371 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886321    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-socket\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886371 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886365    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-credential-socket\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886566 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886436    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-data\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886566 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886524    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-certs\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886566 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886527    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-podinfo\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886833 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886810    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-credential-socket\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886882 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886841    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ba2879-52db-4cc4-b25f-123c48f9556c-istiod-ca-cert\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.886882 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.886853    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-socket\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.888534 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.888511    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-envoy\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.888890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.888875    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-podinfo\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.894671 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.894645    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-token\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:10.894781 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:10.894759    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5sdrc\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-kube-api-access-5sdrc\") pod \"e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") " pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:11.036388 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.036293    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:11.259387 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.259335    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:11.326849 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.326823    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"]
Sep 09 07:15:11.327800 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:15:11.327771    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29ba2879_52db_4cc4_b25f_123c48f9556c.slice/crio-20d3104c188834b6a0528845b2da5644c3a7c7c283296fd9955b3f5aba8ec83a WatchSource:0}: Error finding container 20d3104c188834b6a0528845b2da5644c3a7c7c283296fd9955b3f5aba8ec83a: Status 404 returned error can't find the container with id 20d3104c188834b6a0528845b2da5644c3a7c7c283296fd9955b3f5aba8ec83a
Sep 09 07:15:11.330293 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.330260    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:15:11.330390 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.330340    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:15:11.330706 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.330388    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:15:11.579864 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.579757    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" event={"ID":"29ba2879-52db-4cc4-b25f-123c48f9556c","Type":"ContainerStarted","Data":"247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9"}
Sep 09 07:15:11.579864 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.579815    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" event={"ID":"29ba2879-52db-4cc4-b25f-123c48f9556c","Type":"ContainerStarted","Data":"20d3104c188834b6a0528845b2da5644c3a7c7c283296fd9955b3f5aba8ec83a"}
Sep 09 07:15:11.604457 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:11.604393    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" podStartSLOduration=1.6043698339999999 podStartE2EDuration="1.604369834s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:15:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:15:11.602486945 +0000 UTC m=+1443.972746074" watchObservedRunningTime="2026-09-09 07:15:11.604369834 +0000 UTC m=+1443.974628923"
Sep 09 07:15:12.036647 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:12.036614    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:12.042085 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:12.042061    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:12.584833 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:12.584800    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:12.585863 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:12.585840    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:13.641194 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:13.641154    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:13.738375 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:13.738346    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:13.841137 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:13.841107    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:14.811129 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:14.811091    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:14.911242 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:14.911202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:15.175130 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:15.175098    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:15.304732 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:15.304698    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:15.404684 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:15.404646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:17.065062 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:17.065025    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:22.702082 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:22.702049    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"]
Sep 09 07:15:22.702516 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:22.702277    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv" podUID="4c8db05a-7cff-467b-9efd-add2e56a8b87" containerName="maas-api" containerID="cri-o://00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5" gracePeriod=30
Sep 09 07:15:22.786833 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:22.786782    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:22.883338 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:22.883302    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:22.953402 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:22.953337    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:15:22.974101 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:22.974070    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:23.037161 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.037126    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:23.117212 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.117172    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls\") pod \"4c8db05a-7cff-467b-9efd-add2e56a8b87\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") "
Sep 09 07:15:23.117413 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.117239    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-metrics-certs\") pod \"4c8db05a-7cff-467b-9efd-add2e56a8b87\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") "
Sep 09 07:15:23.117413 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.117406    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8db05a-7cff-467b-9efd-add2e56a8b87-openshift-service-ca\") pod \"4c8db05a-7cff-467b-9efd-add2e56a8b87\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") "
Sep 09 07:15:23.117545 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.117470    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6nmp7\" (UniqueName: \"kubernetes.io/projected/4c8db05a-7cff-467b-9efd-add2e56a8b87-kube-api-access-6nmp7\") pod \"4c8db05a-7cff-467b-9efd-add2e56a8b87\" (UID: \"4c8db05a-7cff-467b-9efd-add2e56a8b87\") "
Sep 09 07:15:23.117800 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.117764    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c8db05a-7cff-467b-9efd-add2e56a8b87-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c8db05a-7cff-467b-9efd-add2e56a8b87" (UID: "4c8db05a-7cff-467b-9efd-add2e56a8b87"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:15:23.119108 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.119080    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c8db05a-7cff-467b-9efd-add2e56a8b87-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:23.119878 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.119846    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "4c8db05a-7cff-467b-9efd-add2e56a8b87" (UID: "4c8db05a-7cff-467b-9efd-add2e56a8b87"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:15:23.119878 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.119854    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c8db05a-7cff-467b-9efd-add2e56a8b87-kube-api-access-6nmp7" (OuterVolumeSpecName: "kube-api-access-6nmp7") pod "4c8db05a-7cff-467b-9efd-add2e56a8b87" (UID: "4c8db05a-7cff-467b-9efd-add2e56a8b87"). InnerVolumeSpecName "kube-api-access-6nmp7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:15:23.120045 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.120026    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c8db05a-7cff-467b-9efd-add2e56a8b87" (UID: "4c8db05a-7cff-467b-9efd-add2e56a8b87"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:15:23.220284 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.220185    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:23.220284 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.220224    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4c8db05a-7cff-467b-9efd-add2e56a8b87-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:23.220284 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.220238    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6nmp7\" (UniqueName: \"kubernetes.io/projected/4c8db05a-7cff-467b-9efd-add2e56a8b87-kube-api-access-6nmp7\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:23.377674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.377639    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:23.647941 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.647850    2563 generic.go:356] "Generic (PLEG): container finished" podID="4c8db05a-7cff-467b-9efd-add2e56a8b87" containerID="00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5" exitCode=0
Sep 09 07:15:23.647941 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.647895    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"
Sep 09 07:15:23.648134 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.647937    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv" event={"ID":"4c8db05a-7cff-467b-9efd-add2e56a8b87","Type":"ContainerDied","Data":"00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5"}
Sep 09 07:15:23.648134 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.647992    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv" event={"ID":"4c8db05a-7cff-467b-9efd-add2e56a8b87","Type":"ContainerDied","Data":"3d9adc9650519ea44f1b82c279d58e6960b8e19c6587a3a786bbcf7533535f92"}
Sep 09 07:15:23.648726 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.648019    2563 scope.go:117] "RemoveContainer" containerID="00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5"
Sep 09 07:15:23.658660 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.658644    2563 scope.go:117] "RemoveContainer" containerID="00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5"
Sep 09 07:15:23.658959 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:15:23.658936    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5\": container with ID starting with 00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5 not found: ID does not exist" containerID="00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5"
Sep 09 07:15:23.659037 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.658973    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5"} err="failed to get container status \"00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5\": rpc error: code = NotFound desc = could not find container \"00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5\": container with ID starting with 00b0bf8eb310fba2fc0d1f93ee9a613113102312f8810c65045887ef7ee1a9f5 not found: ID does not exist"
Sep 09 07:15:23.670473 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.670445    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"]
Sep 09 07:15:23.674707 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:23.674685    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-043f83-77fdf9469f-z4hzv"]
Sep 09 07:15:24.117690 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:24.117658    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c8db05a-7cff-467b-9efd-add2e56a8b87" path="/var/lib/kubelet/pods/4c8db05a-7cff-467b-9efd-add2e56a8b87/volumes"
Sep 09 07:15:29.338463 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:29.338421    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:29.401816 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:29.401767    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:29.656669 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:29.656553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:30.264330 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:30.264294    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:35.720333 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:35.720300    2563 generic.go:356] "Generic (PLEG): container finished" podID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerID="247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9" exitCode=0
Sep 09 07:15:35.720943 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:35.720374    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" event={"ID":"29ba2879-52db-4cc4-b25f-123c48f9556c","Type":"ContainerDied","Data":"247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9"}
Sep 09 07:15:35.721022 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:35.720982    2563 scope.go:117] "RemoveContainer" containerID="247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9"
Sep 09 07:15:35.721903 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:35.721536    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:35.722411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:35.722378    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:15:35.722498 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:35.722467    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:15:35.722554 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:35.722498    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:15:36.727876 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:36.727841    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" event={"ID":"29ba2879-52db-4cc4-b25f-123c48f9556c","Type":"ContainerStarted","Data":"d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2"}
Sep 09 07:15:37.036539 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:37.036452    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:37.036746 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:37.036650    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 07:15:37.036746 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:37.036706    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 07:15:38.036917 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:38.036874    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 07:15:38.037402 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:38.036958    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 07:15:38.873454 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:38.873410    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:39.037060 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:39.037022    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 07:15:39.037441 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:39.037081    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 07:15:39.747973 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:39.747930    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:40.036879 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:40.036795    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 07:15:40.036879 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:40.036847    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 07:15:40.111803 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:40.111764    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:40.285782 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:40.285748    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:40.481988 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:40.481952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:41.037112 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:41.037073    2563 patch_prober.go:28] interesting pod/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 09 07:15:41.037297 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:41.037131    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 09 07:15:41.109533 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:41.109500    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:41.770909 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:41.770876    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:42.042700 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:42.042619    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:42.042916 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:42.042893    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:42.043513 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:42.043491    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:15:43.374973 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:43.374942    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:43.477506 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:43.477474    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:44.506399 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.506362    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"]
Sep 09 07:15:44.507814 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.507784    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c8db05a-7cff-467b-9efd-add2e56a8b87" containerName="maas-api"
Sep 09 07:15:44.507814 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.507812    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c8db05a-7cff-467b-9efd-add2e56a8b87" containerName="maas-api"
Sep 09 07:15:44.508003 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.507923    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c8db05a-7cff-467b-9efd-add2e56a8b87" containerName="maas-api"
Sep 09 07:15:44.511754 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.511729    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"]
Sep 09 07:15:44.512243 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.512216    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.515025 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.515002    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-083e4a-serving-cert\""
Sep 09 07:15:44.534365 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.534329    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:44.636876 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.636838    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:44.645648 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.645622    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.645766 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.645679    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-metrics-certs\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.645853 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.645832    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s8z6\" (UniqueName: \"kubernetes.io/projected/d17119e8-f9e6-4c24-8700-c0c72f64487a-kube-api-access-7s8z6\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.645888 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.645869    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17119e8-f9e6-4c24-8700-c0c72f64487a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.746978 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.746945    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s8z6\" (UniqueName: \"kubernetes.io/projected/d17119e8-f9e6-4c24-8700-c0c72f64487a-kube-api-access-7s8z6\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.746978 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.746986    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17119e8-f9e6-4c24-8700-c0c72f64487a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.747211 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.747010    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.747211 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.747161    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-metrics-certs\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.747725 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.747703    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17119e8-f9e6-4c24-8700-c0c72f64487a-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.749449 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.749419    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-maas-api-tls\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.749613 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.749536    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-metrics-certs\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.752799 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.752770    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:44.754874 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.754852    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s8z6\" (UniqueName: \"kubernetes.io/projected/d17119e8-f9e6-4c24-8700-c0c72f64487a-kube-api-access-7s8z6\") pod \"maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:44.824883 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:44.824797    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:45.101207 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:45.101176    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"]
Sep 09 07:15:45.101791 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:45.101771    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:45.103961 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:15:45.103939    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17119e8_f9e6_4c24_8700_c0c72f64487a.slice/crio-5d42b5810301ae951ceb46541bf794479f134d6fcee3e18c22d9edc782c2eb16 WatchSource:0}: Error finding container 5d42b5810301ae951ceb46541bf794479f134d6fcee3e18c22d9edc782c2eb16: Status 404 returned error can't find the container with id 5d42b5810301ae951ceb46541bf794479f134d6fcee3e18c22d9edc782c2eb16
Sep 09 07:15:45.198770 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:45.198734    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:45.781174 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:45.781140    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh" event={"ID":"d17119e8-f9e6-4c24-8700-c0c72f64487a","Type":"ContainerStarted","Data":"1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466"}
Sep 09 07:15:45.781567 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:45.781198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh" event={"ID":"d17119e8-f9e6-4c24-8700-c0c72f64487a","Type":"ContainerStarted","Data":"5d42b5810301ae951ceb46541bf794479f134d6fcee3e18c22d9edc782c2eb16"}
Sep 09 07:15:45.781567 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:45.781276    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:45.799872 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:45.799821    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh" podStartSLOduration=1.307820976 podStartE2EDuration="1.799803305s" totalImagesPullingTime="491.982329ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:15:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:15:45.798349204 +0000 UTC m=+1478.168608293" watchObservedRunningTime="2026-09-09 07:15:45.799803305 +0000 UTC m=+1478.170062392"
Sep 09 07:15:46.352161 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:46.352128    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:46.442569 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:46.442529    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:46.508463 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:46.508430    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:46.577214 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:46.577185    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:46.979975 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:46.979947    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:51.792329 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:51.792298    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:15:53.280205 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:53.280162    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:53.377269 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:53.377234    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:53.548947 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:53.548859    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"]
Sep 09 07:15:53.549179 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:53.549126    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy" containerID="cri-o://867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82" gracePeriod=30
Sep 09 07:15:53.900147 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:53.900064    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:54.211829 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:54.211793    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:15:58.797875 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.797851    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:15:58.854111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.854024    2563 generic.go:356] "Generic (PLEG): container finished" podID="40df5d19-ec78-4f40-a782-a7afad941d53" containerID="867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82" exitCode=0
Sep 09 07:15:58.854111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.854079    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"
Sep 09 07:15:58.854283 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.854110    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" event={"ID":"40df5d19-ec78-4f40-a782-a7afad941d53","Type":"ContainerDied","Data":"867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82"}
Sep 09 07:15:58.854283 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.854169    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8" event={"ID":"40df5d19-ec78-4f40-a782-a7afad941d53","Type":"ContainerDied","Data":"13734a89b294ab480731b3e9b3ccd988ed11dd6b7360c4ffec1b9f95f4c60e5d"}
Sep 09 07:15:58.854283 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.854193    2563 scope.go:117] "RemoveContainer" containerID="867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82"
Sep 09 07:15:58.863479 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.863461    2563 scope.go:117] "RemoveContainer" containerID="80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860"
Sep 09 07:15:58.871832 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.871817    2563 scope.go:117] "RemoveContainer" containerID="867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82"
Sep 09 07:15:58.872080 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:15:58.872061    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82\": container with ID starting with 867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82 not found: ID does not exist" containerID="867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82"
Sep 09 07:15:58.872123 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.872088    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82"} err="failed to get container status \"867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82\": rpc error: code = NotFound desc = could not find container \"867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82\": container with ID starting with 867a2778bc03517677d7e42eeb524acd1cbc5c7eb634edd23e2bc65d41bfad82 not found: ID does not exist"
Sep 09 07:15:58.872123 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.872109    2563 scope.go:117] "RemoveContainer" containerID="80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860"
Sep 09 07:15:58.872343 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:15:58.872324    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860\": container with ID starting with 80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860 not found: ID does not exist" containerID="80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860"
Sep 09 07:15:58.872391 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.872350    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860"} err="failed to get container status \"80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860\": rpc error: code = NotFound desc = could not find container \"80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860\": container with ID starting with 80cae8a638c27338d354a259b3dcdcf773175f422762849cdb46f5535bd91860 not found: ID does not exist"
Sep 09 07:15:58.895839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.895815    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40df5d19-ec78-4f40-a782-a7afad941d53-istiod-ca-cert\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.895925 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.895849    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gr69\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-kube-api-access-4gr69\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.895925 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.895890    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-credential-socket\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.896022 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.895999    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-istio-token\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.896076 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896044    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40df5d19-ec78-4f40-a782-a7afad941d53-istio-podinfo\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.896125 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896116    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-envoy\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.896200 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896184    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-data\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.896251 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896115    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:15:58.896251 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896184    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40df5d19-ec78-4f40-a782-a7afad941d53-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:15:58.896356 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896252    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-certs\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.896356 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896308    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-socket\") pod \"40df5d19-ec78-4f40-a782-a7afad941d53\" (UID: \"40df5d19-ec78-4f40-a782-a7afad941d53\") "
Sep 09 07:15:58.896467 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896410    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-data" (OuterVolumeSpecName: "istio-data") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:15:58.896527 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896501    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:15:58.896658 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896633    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:15:58.896841 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896820    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:58.896901 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896847    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:58.896901 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896862    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40df5d19-ec78-4f40-a782-a7afad941d53-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:58.896901 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896876    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:58.896901 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.896890    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:58.898208 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.898184    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40df5d19-ec78-4f40-a782-a7afad941d53-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:15:58.898286 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.898257    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-kube-api-access-4gr69" (OuterVolumeSpecName: "kube-api-access-4gr69") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "kube-api-access-4gr69". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:15:58.898518 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.898493    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:15:58.898518 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.898496    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-istio-token" (OuterVolumeSpecName: "istio-token") pod "40df5d19-ec78-4f40-a782-a7afad941d53" (UID: "40df5d19-ec78-4f40-a782-a7afad941d53"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:15:58.997914 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.997883    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gr69\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-kube-api-access-4gr69\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:58.997914 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.997912    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40df5d19-ec78-4f40-a782-a7afad941d53-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:58.998107 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.997924    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40df5d19-ec78-4f40-a782-a7afad941d53-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:58.998107 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:58.997938    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40df5d19-ec78-4f40-a782-a7afad941d53-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:15:59.176540 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:59.176502    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"]
Sep 09 07:15:59.180335 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:15:59.180309    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-d8398c-openshift-default-89dbd9b56-mn5c8"]
Sep 09 07:16:00.118010 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:00.117976    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" path="/var/lib/kubelet/pods/40df5d19-ec78-4f40-a782-a7afad941d53/volumes"
Sep 09 07:16:04.523411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.523375    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"]
Sep 09 07:16:04.523888 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.523649    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6" podUID="e7c088fe-b76b-40b1-842e-7b664c7cd5db" containerName="maas-api" containerID="cri-o://27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1" gracePeriod=30
Sep 09 07:16:04.557478 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.557447    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:04.655180 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.655146    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:04.755448 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.755412    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:04.779021 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.778970    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:16:04.854486 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.854452    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7c088fe-b76b-40b1-842e-7b664c7cd5db-openshift-service-ca\") pod \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") "
Sep 09 07:16:04.854646 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.854508    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cjgjz\" (UniqueName: \"kubernetes.io/projected/e7c088fe-b76b-40b1-842e-7b664c7cd5db-kube-api-access-cjgjz\") pod \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") "
Sep 09 07:16:04.854730 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.854709    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-metrics-certs\") pod \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") "
Sep 09 07:16:04.854853 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.854832    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e7c088fe-b76b-40b1-842e-7b664c7cd5db-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e7c088fe-b76b-40b1-842e-7b664c7cd5db" (UID: "e7c088fe-b76b-40b1-842e-7b664c7cd5db"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:16:04.855129 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.855106    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e7c088fe-b76b-40b1-842e-7b664c7cd5db-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:16:04.856685 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.856652    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e7c088fe-b76b-40b1-842e-7b664c7cd5db-kube-api-access-cjgjz" (OuterVolumeSpecName: "kube-api-access-cjgjz") pod "e7c088fe-b76b-40b1-842e-7b664c7cd5db" (UID: "e7c088fe-b76b-40b1-842e-7b664c7cd5db"). InnerVolumeSpecName "kube-api-access-cjgjz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:16:04.856784 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.856737    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e7c088fe-b76b-40b1-842e-7b664c7cd5db" (UID: "e7c088fe-b76b-40b1-842e-7b664c7cd5db"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:16:04.857068 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.857043    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:04.899921 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.899887    2563 generic.go:356] "Generic (PLEG): container finished" podID="e7c088fe-b76b-40b1-842e-7b664c7cd5db" containerID="27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1" exitCode=0
Sep 09 07:16:04.900078 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.899933    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"
Sep 09 07:16:04.900078 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.899965    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6" event={"ID":"e7c088fe-b76b-40b1-842e-7b664c7cd5db","Type":"ContainerDied","Data":"27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1"}
Sep 09 07:16:04.900078 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.900019    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6" event={"ID":"e7c088fe-b76b-40b1-842e-7b664c7cd5db","Type":"ContainerDied","Data":"b1248d66a3c40062fd4388ae45ff919f708b2f9b2ba6793a8f7197ac659a2a2d"}
Sep 09 07:16:04.900078 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.900041    2563 scope.go:117] "RemoveContainer" containerID="27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1"
Sep 09 07:16:04.910056 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.910040    2563 scope.go:117] "RemoveContainer" containerID="27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1"
Sep 09 07:16:04.910360 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:16:04.910337    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1\": container with ID starting with 27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1 not found: ID does not exist" containerID="27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1"
Sep 09 07:16:04.910439 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.910371    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1"} err="failed to get container status \"27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1\": rpc error: code = NotFound desc = could not find container \"27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1\": container with ID starting with 27215cd15c2b3209c61f85ba312252da5df053e5912fc4ea1d920d65097f1bf1 not found: ID does not exist"
Sep 09 07:16:04.956036 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.956002    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls\") pod \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\" (UID: \"e7c088fe-b76b-40b1-842e-7b664c7cd5db\") "
Sep 09 07:16:04.956268 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.956252    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cjgjz\" (UniqueName: \"kubernetes.io/projected/e7c088fe-b76b-40b1-842e-7b664c7cd5db-kube-api-access-cjgjz\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:16:04.956366 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.956270    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:16:04.958102 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:04.958077    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e7c088fe-b76b-40b1-842e-7b664c7cd5db" (UID: "e7c088fe-b76b-40b1-842e-7b664c7cd5db"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:16:05.057401 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:05.057313    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e7c088fe-b76b-40b1-842e-7b664c7cd5db-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:16:05.224031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:05.224002    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"]
Sep 09 07:16:05.229872 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:05.229847    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w2-1128ac-5767f468c8-d5nw6"]
Sep 09 07:16:05.235774 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:05.235752    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:05.512111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:05.512082    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:06.119147 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:06.119117    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e7c088fe-b76b-40b1-842e-7b664c7cd5db" path="/var/lib/kubelet/pods/e7c088fe-b76b-40b1-842e-7b664c7cd5db/volumes"
Sep 09 07:16:08.213282 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:08.213256    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:16:08.215391 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:08.215368    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:16:10.374407 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:10.374372    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:11.128454 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:11.128417    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:11.196206 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:11.196173    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:11.425503 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:11.425476    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:11.921273 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:11.921179    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:17.731165 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:17.731130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:21.934871 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:21.934800    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:22.538573 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.538540    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"]
Sep 09 07:16:22.538842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.538798    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb" podUID="2e134e22-c8ff-455b-bee6-691f6d13e7fc" containerName="maas-api" containerID="cri-o://8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13" gracePeriod=30
Sep 09 07:16:22.573167 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.573130    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:22.671429 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.671387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:22.771071 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.771045    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:22.792701 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.792642    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:16:22.873549 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.873514    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:22.941724 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.941689    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e134e22-c8ff-455b-bee6-691f6d13e7fc-openshift-service-ca\") pod \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") "
Sep 09 07:16:22.941724 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.941728    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-maas-api-tls\") pod \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") "
Sep 09 07:16:22.942234 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.941758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-metrics-certs\") pod \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") "
Sep 09 07:16:22.942234 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.941807    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nk6k\" (UniqueName: \"kubernetes.io/projected/2e134e22-c8ff-455b-bee6-691f6d13e7fc-kube-api-access-8nk6k\") pod \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\" (UID: \"2e134e22-c8ff-455b-bee6-691f6d13e7fc\") "
Sep 09 07:16:22.942234 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.942133    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2e134e22-c8ff-455b-bee6-691f6d13e7fc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2e134e22-c8ff-455b-bee6-691f6d13e7fc" (UID: "2e134e22-c8ff-455b-bee6-691f6d13e7fc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:16:22.945348 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.945321    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2e134e22-c8ff-455b-bee6-691f6d13e7fc-kube-api-access-8nk6k" (OuterVolumeSpecName: "kube-api-access-8nk6k") pod "2e134e22-c8ff-455b-bee6-691f6d13e7fc" (UID: "2e134e22-c8ff-455b-bee6-691f6d13e7fc"). InnerVolumeSpecName "kube-api-access-8nk6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:16:22.946931 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.946894    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2e134e22-c8ff-455b-bee6-691f6d13e7fc" (UID: "2e134e22-c8ff-455b-bee6-691f6d13e7fc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:16:22.948109 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.948082    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2e134e22-c8ff-455b-bee6-691f6d13e7fc" (UID: "2e134e22-c8ff-455b-bee6-691f6d13e7fc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:16:22.998820 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.998792    2563 generic.go:356] "Generic (PLEG): container finished" podID="2e134e22-c8ff-455b-bee6-691f6d13e7fc" containerID="8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13" exitCode=0
Sep 09 07:16:22.998972 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.998837    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"
Sep 09 07:16:22.998972 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.998863    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb" event={"ID":"2e134e22-c8ff-455b-bee6-691f6d13e7fc","Type":"ContainerDied","Data":"8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13"}
Sep 09 07:16:22.998972 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.998928    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb" event={"ID":"2e134e22-c8ff-455b-bee6-691f6d13e7fc","Type":"ContainerDied","Data":"65161669a7d204b1c4c5e7dc87138ca6c9a7b6ddadfd7e9688f049f9e47f8cf6"}
Sep 09 07:16:22.998972 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:22.998949    2563 scope.go:117] "RemoveContainer" containerID="8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13"
Sep 09 07:16:23.008518 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.008505    2563 scope.go:117] "RemoveContainer" containerID="8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13"
Sep 09 07:16:23.008803 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:16:23.008782    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13\": container with ID starting with 8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13 not found: ID does not exist" containerID="8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13"
Sep 09 07:16:23.008897 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.008810    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13"} err="failed to get container status \"8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13\": rpc error: code = NotFound desc = could not find container \"8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13\": container with ID starting with 8b9c86ca20c2368f1ad6284522d4d33f1a2059d81be7a5b59d74d7e334645a13 not found: ID does not exist"
Sep 09 07:16:23.022158 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.022133    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"]
Sep 09 07:16:23.031250 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.031220    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1d8843-854dffb869-2s2rb"]
Sep 09 07:16:23.043530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.043469    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nk6k\" (UniqueName: \"kubernetes.io/projected/2e134e22-c8ff-455b-bee6-691f6d13e7fc-kube-api-access-8nk6k\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:16:23.043530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.043495    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2e134e22-c8ff-455b-bee6-691f6d13e7fc-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:16:23.043530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.043510    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:16:23.043530 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.043524    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e134e22-c8ff-455b-bee6-691f6d13e7fc-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:16:23.259900 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:23.259871    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:24.117808 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:24.117776    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2e134e22-c8ff-455b-bee6-691f6d13e7fc" path="/var/lib/kubelet/pods/2e134e22-c8ff-455b-bee6-691f6d13e7fc/volumes"
Sep 09 07:16:25.856405 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:25.856373    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:28.947091 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:28.947058    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:29.047929 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:29.047897    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:29.259376 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:29.259299    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:58.215515 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.215481    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:16:58.558223 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.558118    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"]
Sep 09 07:16:58.558851 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.558826    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy"
Sep 09 07:16:58.559046 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559025    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy"
Sep 09 07:16:58.559171 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559054    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2e134e22-c8ff-455b-bee6-691f6d13e7fc" containerName="maas-api"
Sep 09 07:16:58.559171 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559064    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="2e134e22-c8ff-455b-bee6-691f6d13e7fc" containerName="maas-api"
Sep 09 07:16:58.559171 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559110    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy"
Sep 09 07:16:58.559171 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559119    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy"
Sep 09 07:16:58.559171 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559133    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e7c088fe-b76b-40b1-842e-7b664c7cd5db" containerName="maas-api"
Sep 09 07:16:58.559171 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559141    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e7c088fe-b76b-40b1-842e-7b664c7cd5db" containerName="maas-api"
Sep 09 07:16:58.559444 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559257    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy"
Sep 09 07:16:58.559444 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559273    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="2e134e22-c8ff-455b-bee6-691f6d13e7fc" containerName="maas-api"
Sep 09 07:16:58.559444 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.559284    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e7c088fe-b76b-40b1-842e-7b664c7cd5db" containerName="maas-api"
Sep 09 07:16:58.562499 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.562479    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.565274 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.565254    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-36a83e-openshift-default-dockercfg-wwgcc\""
Sep 09 07:16:58.576700 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.576674    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"]
Sep 09 07:16:58.716823 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.716777    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-socket\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.717018 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.716860    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzlq4\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-kube-api-access-hzlq4\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.717018 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.716898    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-token\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.717018 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.716930    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-podinfo\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.717018 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.716949    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-envoy\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.717018 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.716967    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-certs\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.717018 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.716994    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-data\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.717247 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.717023    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-credential-socket\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.717247 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.717047    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dea144c-972a-4ed7-a4b3-22d5db87659e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818139 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818056    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzlq4\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-kube-api-access-hzlq4\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818139 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818113    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-token\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818346 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818142    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-podinfo\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818346 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818165    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-envoy\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818346 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818188    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-certs\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818346 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818215    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-data\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818346 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818259    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-credential-socket\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818346 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818298    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dea144c-972a-4ed7-a4b3-22d5db87659e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818699 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818377    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-socket\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818802 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818777    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-certs\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818805    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-data\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.818951 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818885    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-credential-socket\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.819012 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.818964    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-socket\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.819206 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.819185    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dea144c-972a-4ed7-a4b3-22d5db87659e-istiod-ca-cert\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.820634 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.820616    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-envoy\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.820773 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.820756    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-podinfo\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.826295 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.826273    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-token\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.826439 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.826418    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzlq4\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-kube-api-access-hzlq4\") pod \"e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") " pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:58.875737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:58.875699    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:16:59.154045 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:59.154023    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"]
Sep 09 07:16:59.156550 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:16:59.156522    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dea144c_972a_4ed7_a4b3_22d5db87659e.slice/crio-06c4273c1fbf270d0b93f14eeb2c7617116c134ad0a187c1deb7d8c045518260 WatchSource:0}: Error finding container 06c4273c1fbf270d0b93f14eeb2c7617116c134ad0a187c1deb7d8c045518260: Status 404 returned error can't find the container with id 06c4273c1fbf270d0b93f14eeb2c7617116c134ad0a187c1deb7d8c045518260
Sep 09 07:16:59.158968 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:59.158927    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:16:59.159048 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:59.159021    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:16:59.159091 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:59.159064    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:16:59.190362 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:59.190332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" event={"ID":"9dea144c-972a-4ed7-a4b3-22d5db87659e","Type":"ContainerStarted","Data":"06c4273c1fbf270d0b93f14eeb2c7617116c134ad0a187c1deb7d8c045518260"}
Sep 09 07:16:59.583432 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:16:59.583389    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:00.197569 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:00.197531    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" event={"ID":"9dea144c-972a-4ed7-a4b3-22d5db87659e","Type":"ContainerStarted","Data":"080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec"}
Sep 09 07:17:00.220784 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:00.220731    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" podStartSLOduration=2.220715421 podStartE2EDuration="2.220715421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:17:00.218719677 +0000 UTC m=+1552.588978765" watchObservedRunningTime="2026-09-09 07:17:00.220715421 +0000 UTC m=+1552.590974508"
Sep 09 07:17:00.542784 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:00.542706    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:00.876619 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:00.876523    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:00.882432 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:00.882404    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:01.203408 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:01.203372    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:01.204458 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:01.204441    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:02.211775 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:02.211743    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:02.312099 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:02.312070    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:03.384200 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:03.384171    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:03.483683 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:03.483648    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:03.680912 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:03.680877    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:03.884903 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:03.884871    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:03.986121 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:03.986005    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:20.313507 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:20.313468    2563 generic.go:356] "Generic (PLEG): container finished" podID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerID="080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec" exitCode=0
Sep 09 07:17:20.313955 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:20.313539    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" event={"ID":"9dea144c-972a-4ed7-a4b3-22d5db87659e","Type":"ContainerDied","Data":"080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec"}
Sep 09 07:17:20.313999 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:20.313984    2563 scope.go:117] "RemoveContainer" containerID="080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec"
Sep 09 07:17:20.315281 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:20.315257    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:17:20.315342 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:20.315329    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:17:20.315381 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:20.315362    2563 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 09 07:17:21.320826 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:21.320792    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" event={"ID":"9dea144c-972a-4ed7-a4b3-22d5db87659e","Type":"ContainerStarted","Data":"5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251"}
Sep 09 07:17:21.876807 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:21.876767    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:21.877037 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:21.877010    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 07:17:21.877091 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:21.877076    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 07:17:22.876998 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:22.876963    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 07:17:22.877448 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:22.877021    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 07:17:23.876426 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:23.876387    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 07:17:23.876621 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:23.876443    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 07:17:24.876335 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:24.876286    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 07:17:24.876872 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:24.876354    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 07:17:25.877019 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:25.876980    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 09 07:17:25.877467 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:25.877046    2563 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 09 07:17:26.906737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:26.906677    2563 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:26.907192 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:26.907103    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:26.907973 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:26.907951    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:27.305634 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:27.305570    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:33.964757 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:33.964713    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:34.036269 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:34.036236    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:34.104993 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:34.104952    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:34.183201 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:34.183169    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:34.243676 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:34.243575    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:34.539541 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:34.539457    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:40.272678 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:40.272646    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:40.339836 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:40.339804    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:40.516771 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:40.516737    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"]
Sep 09 07:17:40.516989 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:40.516962    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy" containerID="cri-o://5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251" gracePeriod=30
Sep 09 07:17:40.533790 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:40.533718    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:41.910280 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:41.910240    2563 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 09 07:17:41.910715 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:41.910310    2563 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 09 07:17:44.758065 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:44.758028    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:45.774434 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.774412    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:45.903289 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903206    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzlq4\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-kube-api-access-hzlq4\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903289 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903267    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-data\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903478 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903298    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-socket\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903478 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903451    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-token\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903559 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903509    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-credential-socket\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903559 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903537    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:17:45.903681 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903579    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-data" (OuterVolumeSpecName: "istio-data") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:17:45.903681 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903647    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-certs\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903760 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903731    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-podinfo\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903758    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dea144c-972a-4ed7-a4b3-22d5db87659e-istiod-ca-cert\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903786    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-envoy\") pod \"9dea144c-972a-4ed7-a4b3-22d5db87659e\" (UID: \"9dea144c-972a-4ed7-a4b3-22d5db87659e\") "
Sep 09 07:17:45.903805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903774    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:17:45.903962 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.903914    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:17:45.904182 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.904159    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:45.904182 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.904183    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:45.904373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.904205    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:45.904373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.904216    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:45.904373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.904158    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dea144c-972a-4ed7-a4b3-22d5db87659e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:17:45.905835 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.905805    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:17:45.905929 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.905839    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:17:45.905985 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.905968    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-kube-api-access-hzlq4" (OuterVolumeSpecName: "kube-api-access-hzlq4") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "kube-api-access-hzlq4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:17:45.906026 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:45.905994    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-token" (OuterVolumeSpecName: "istio-token") pod "9dea144c-972a-4ed7-a4b3-22d5db87659e" (UID: "9dea144c-972a-4ed7-a4b3-22d5db87659e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:17:46.005288 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.005253    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:46.005288 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.005282    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:46.005288 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.005291    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9dea144c-972a-4ed7-a4b3-22d5db87659e-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:46.005528 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.005300    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9dea144c-972a-4ed7-a4b3-22d5db87659e-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:46.005528 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.005308    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzlq4\" (UniqueName: \"kubernetes.io/projected/9dea144c-972a-4ed7-a4b3-22d5db87659e-kube-api-access-hzlq4\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:17:46.456027 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.455987    2563 generic.go:356] "Generic (PLEG): container finished" podID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerID="5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251" exitCode=0
Sep 09 07:17:46.456216 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.456035    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"
Sep 09 07:17:46.456216 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.456035    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" event={"ID":"9dea144c-972a-4ed7-a4b3-22d5db87659e","Type":"ContainerDied","Data":"5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251"}
Sep 09 07:17:46.456216 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.456109    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5" event={"ID":"9dea144c-972a-4ed7-a4b3-22d5db87659e","Type":"ContainerDied","Data":"06c4273c1fbf270d0b93f14eeb2c7617116c134ad0a187c1deb7d8c045518260"}
Sep 09 07:17:46.456216 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.456141    2563 scope.go:117] "RemoveContainer" containerID="5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251"
Sep 09 07:17:46.465491 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.465481    2563 scope.go:117] "RemoveContainer" containerID="080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec"
Sep 09 07:17:46.474348 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.474332    2563 scope.go:117] "RemoveContainer" containerID="5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251"
Sep 09 07:17:46.474674 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:17:46.474635    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251\": container with ID starting with 5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251 not found: ID does not exist" containerID="5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251"
Sep 09 07:17:46.474731 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.474683    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251"} err="failed to get container status \"5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251\": rpc error: code = NotFound desc = could not find container \"5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251\": container with ID starting with 5901ef7186c3d6cd602b4bcc5e9ac4edc0d8256f02278f83f47ae87cd59a0251 not found: ID does not exist"
Sep 09 07:17:46.474731 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.474701    2563 scope.go:117] "RemoveContainer" containerID="080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec"
Sep 09 07:17:46.474731 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.474702    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"]
Sep 09 07:17:46.474981 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:17:46.474964    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec\": container with ID starting with 080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec not found: ID does not exist" containerID="080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec"
Sep 09 07:17:46.475030 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.474985    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec"} err="failed to get container status \"080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec\": rpc error: code = NotFound desc = could not find container \"080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec\": container with ID starting with 080f5468b53a1b9b2e06e4ee6f2270c47bb90270d30563d232ee77556cc1e3ec not found: ID does not exist"
Sep 09 07:17:46.478816 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:46.478793    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-36a83e-openshift-default-6f49b7f67b-rmqc5"]
Sep 09 07:17:48.120333 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:48.120284    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" path="/var/lib/kubelet/pods/9dea144c-972a-4ed7-a4b3-22d5db87659e/volumes"
Sep 09 07:17:59.635146 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:59.635112    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:59.733506 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:59.733478    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:59.838009 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:59.837972    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:17:59.961242 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:17:59.961202    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:00.284280 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:00.284188    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:05.339753 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:05.339720    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:41.454812 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:41.454778    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:42.209301 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:42.209266    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:47.806441 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:47.806400    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:47.906433 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:47.906387    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:48.060830 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:48.060748    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"]
Sep 09 07:18:48.061048 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:48.061022    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy" containerID="cri-o://d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2" gracePeriod=30
Sep 09 07:18:48.086026 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:48.085998    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:48.590369 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:48.590335    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:53.319763 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.319742    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:18:53.381783 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.381697    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-credential-socket\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.381783 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.381740    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-socket\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.381783 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.381779    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-envoy\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.382034 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.381825    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5sdrc\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-kube-api-access-5sdrc\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.382034 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.381881    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-certs\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.382034 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.381918    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-data\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.382034 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382030    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:18:53.382250 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382041    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-podinfo\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.382250 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382094    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-token\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.382250 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382109    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:18:53.382250 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382134    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ba2879-52db-4cc4-b25f-123c48f9556c-istiod-ca-cert\") pod \"29ba2879-52db-4cc4-b25f-123c48f9556c\" (UID: \"29ba2879-52db-4cc4-b25f-123c48f9556c\") "
Sep 09 07:18:53.382471 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382261    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-data" (OuterVolumeSpecName: "istio-data") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:18:53.382471 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382305    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:18:53.382650 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382581    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29ba2879-52db-4cc4-b25f-123c48f9556c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:18:53.382650 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382617    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.382650 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382636    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.382650 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382648    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.382881 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.382661    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.384186 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.384162    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:18:53.384186 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.384174    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-kube-api-access-5sdrc" (OuterVolumeSpecName: "kube-api-access-5sdrc") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "kube-api-access-5sdrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:18:53.384398 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.384379    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:18:53.384463 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.384434    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-token" (OuterVolumeSpecName: "istio-token") pod "29ba2879-52db-4cc4-b25f-123c48f9556c" (UID: "29ba2879-52db-4cc4-b25f-123c48f9556c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:18:53.483460 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.483426    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/29ba2879-52db-4cc4-b25f-123c48f9556c-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.483460 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.483452    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.483460 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.483461    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5sdrc\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-kube-api-access-5sdrc\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.483460 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.483470    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.483741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.483478    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/29ba2879-52db-4cc4-b25f-123c48f9556c-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:53.811465 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.811431    2563 generic.go:356] "Generic (PLEG): container finished" podID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerID="d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2" exitCode=0
Sep 09 07:18:53.811685 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.811477    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"
Sep 09 07:18:53.811685 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.811521    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" event={"ID":"29ba2879-52db-4cc4-b25f-123c48f9556c","Type":"ContainerDied","Data":"d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2"}
Sep 09 07:18:53.811685 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.811580    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt" event={"ID":"29ba2879-52db-4cc4-b25f-123c48f9556c","Type":"ContainerDied","Data":"20d3104c188834b6a0528845b2da5644c3a7c7c283296fd9955b3f5aba8ec83a"}
Sep 09 07:18:53.811685 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.811629    2563 scope.go:117] "RemoveContainer" containerID="d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2"
Sep 09 07:18:53.821403 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.821386    2563 scope.go:117] "RemoveContainer" containerID="247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9"
Sep 09 07:18:53.829485 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.829469    2563 scope.go:117] "RemoveContainer" containerID="d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2"
Sep 09 07:18:53.829780 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:18:53.829762    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2\": container with ID starting with d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2 not found: ID does not exist" containerID="d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2"
Sep 09 07:18:53.829839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.829789    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2"} err="failed to get container status \"d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2\": rpc error: code = NotFound desc = could not find container \"d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2\": container with ID starting with d7c4f4f31d484203b74c0f870c96ed5f279796907d5d196bff56f60a88ae7ec2 not found: ID does not exist"
Sep 09 07:18:53.829839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.829806    2563 scope.go:117] "RemoveContainer" containerID="247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9"
Sep 09 07:18:53.830066 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:18:53.830046    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9\": container with ID starting with 247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9 not found: ID does not exist" containerID="247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9"
Sep 09 07:18:53.830122 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.830077    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9"} err="failed to get container status \"247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9\": rpc error: code = NotFound desc = could not find container \"247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9\": container with ID starting with 247dcfae7baaec43a529736485413948ded1c3103b2ab3086ea406deb81e19d9 not found: ID does not exist"
Sep 09 07:18:53.834877 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.834855    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"]
Sep 09 07:18:53.841236 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:53.841212    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-bea943-openshift-default-75b686878b-2k2bt"]
Sep 09 07:18:54.124974 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:54.124888    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" path="/var/lib/kubelet/pods/29ba2879-52db-4cc4-b25f-123c48f9556c/volumes"
Sep 09 07:18:58.928264 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:58.928226    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"]
Sep 09 07:18:58.928708 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:58.928443    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh" podUID="d17119e8-f9e6-4c24-8700-c0c72f64487a" containerName="maas-api" containerID="cri-o://1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466" gracePeriod=30
Sep 09 07:18:58.957394 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:58.957367    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:59.057930 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.057879    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:59.156921 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.156886    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:59.184050 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.183997    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:18:59.229127 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.229089    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:59.241674 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.241647    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17119e8-f9e6-4c24-8700-c0c72f64487a-openshift-service-ca\") pod \"d17119e8-f9e6-4c24-8700-c0c72f64487a\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") "
Sep 09 07:18:59.241818 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.241699    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-metrics-certs\") pod \"d17119e8-f9e6-4c24-8700-c0c72f64487a\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") "
Sep 09 07:18:59.241818 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.241721    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-maas-api-tls\") pod \"d17119e8-f9e6-4c24-8700-c0c72f64487a\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") "
Sep 09 07:18:59.241818 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.241750    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7s8z6\" (UniqueName: \"kubernetes.io/projected/d17119e8-f9e6-4c24-8700-c0c72f64487a-kube-api-access-7s8z6\") pod \"d17119e8-f9e6-4c24-8700-c0c72f64487a\" (UID: \"d17119e8-f9e6-4c24-8700-c0c72f64487a\") "
Sep 09 07:18:59.241985 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.241955    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d17119e8-f9e6-4c24-8700-c0c72f64487a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "d17119e8-f9e6-4c24-8700-c0c72f64487a" (UID: "d17119e8-f9e6-4c24-8700-c0c72f64487a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:18:59.242074 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.242056    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d17119e8-f9e6-4c24-8700-c0c72f64487a-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:59.243770 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.243746    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "d17119e8-f9e6-4c24-8700-c0c72f64487a" (UID: "d17119e8-f9e6-4c24-8700-c0c72f64487a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:18:59.243874 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.243771    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "d17119e8-f9e6-4c24-8700-c0c72f64487a" (UID: "d17119e8-f9e6-4c24-8700-c0c72f64487a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:18:59.243874 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.243788    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d17119e8-f9e6-4c24-8700-c0c72f64487a-kube-api-access-7s8z6" (OuterVolumeSpecName: "kube-api-access-7s8z6") pod "d17119e8-f9e6-4c24-8700-c0c72f64487a" (UID: "d17119e8-f9e6-4c24-8700-c0c72f64487a"). InnerVolumeSpecName "kube-api-access-7s8z6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:18:59.343255 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.343219    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:59.343255 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.343250    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d17119e8-f9e6-4c24-8700-c0c72f64487a-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:59.343255 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.343260    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7s8z6\" (UniqueName: \"kubernetes.io/projected/d17119e8-f9e6-4c24-8700-c0c72f64487a-kube-api-access-7s8z6\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:18:59.541715 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.541632    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:18:59.844332 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.844252    2563 generic.go:356] "Generic (PLEG): container finished" podID="d17119e8-f9e6-4c24-8700-c0c72f64487a" containerID="1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466" exitCode=0
Sep 09 07:18:59.844332 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.844294    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"
Sep 09 07:18:59.844510 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.844332    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh" event={"ID":"d17119e8-f9e6-4c24-8700-c0c72f64487a","Type":"ContainerDied","Data":"1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466"}
Sep 09 07:18:59.844510 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.844387    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh" event={"ID":"d17119e8-f9e6-4c24-8700-c0c72f64487a","Type":"ContainerDied","Data":"5d42b5810301ae951ceb46541bf794479f134d6fcee3e18c22d9edc782c2eb16"}
Sep 09 07:18:59.844510 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.844416    2563 scope.go:117] "RemoveContainer" containerID="1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466"
Sep 09 07:18:59.854163 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.854141    2563 scope.go:117] "RemoveContainer" containerID="1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466"
Sep 09 07:18:59.854440 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:18:59.854422    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466\": container with ID starting with 1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466 not found: ID does not exist" containerID="1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466"
Sep 09 07:18:59.854513 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.854448    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466"} err="failed to get container status \"1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466\": rpc error: code = NotFound desc = could not find container \"1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466\": container with ID starting with 1b1fcf8878b6d051738306eba886099a4cf491d3978c6b1d2556fe738ec1f466 not found: ID does not exist"
Sep 09 07:18:59.866344 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.866316    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"]
Sep 09 07:18:59.872918 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:18:59.872899    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-083e4a-5b7546758d-tf2mh"]
Sep 09 07:19:00.117803 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:00.117727    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d17119e8-f9e6-4c24-8700-c0c72f64487a" path="/var/lib/kubelet/pods/d17119e8-f9e6-4c24-8700-c0c72f64487a/volumes"
Sep 09 07:19:01.440346 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:01.440310    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:05.361820 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:05.361785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:05.463566 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:05.463533    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:05.648052 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:05.647969    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:16.437720 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.437689    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"]
Sep 09 07:19:16.438267 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.437891    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz" podUID="04d2e308-9a02-4a1b-aa34-b94adc09dacd" containerName="maas-api" containerID="cri-o://d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787" gracePeriod=30
Sep 09 07:19:16.471132 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.471104    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:16.564709 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.564676    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:16.632033 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.632004    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:16.687900 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.687848    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:19:16.703427 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.703405    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:16.811323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.811280    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-metrics-certs\") pod \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") "
Sep 09 07:19:16.811509 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.811377    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mdqsx\" (UniqueName: \"kubernetes.io/projected/04d2e308-9a02-4a1b-aa34-b94adc09dacd-kube-api-access-mdqsx\") pod \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") "
Sep 09 07:19:16.811509 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.811429    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d2e308-9a02-4a1b-aa34-b94adc09dacd-openshift-service-ca\") pod \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") "
Sep 09 07:19:16.811509 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.811446    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls\") pod \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\" (UID: \"04d2e308-9a02-4a1b-aa34-b94adc09dacd\") "
Sep 09 07:19:16.811839 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.811803    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04d2e308-9a02-4a1b-aa34-b94adc09dacd-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04d2e308-9a02-4a1b-aa34-b94adc09dacd" (UID: "04d2e308-9a02-4a1b-aa34-b94adc09dacd"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:19:16.813617 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.813554    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04d2e308-9a02-4a1b-aa34-b94adc09dacd" (UID: "04d2e308-9a02-4a1b-aa34-b94adc09dacd"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:19:16.813725 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.813640    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "04d2e308-9a02-4a1b-aa34-b94adc09dacd" (UID: "04d2e308-9a02-4a1b-aa34-b94adc09dacd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:19:16.813883 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.813866    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04d2e308-9a02-4a1b-aa34-b94adc09dacd-kube-api-access-mdqsx" (OuterVolumeSpecName: "kube-api-access-mdqsx") pod "04d2e308-9a02-4a1b-aa34-b94adc09dacd" (UID: "04d2e308-9a02-4a1b-aa34-b94adc09dacd"). InnerVolumeSpecName "kube-api-access-mdqsx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:19:16.912868 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.912820    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04d2e308-9a02-4a1b-aa34-b94adc09dacd-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:16.912868 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.912863    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:16.912868 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.912873    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04d2e308-9a02-4a1b-aa34-b94adc09dacd-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:16.912868 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.912880    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mdqsx\" (UniqueName: \"kubernetes.io/projected/04d2e308-9a02-4a1b-aa34-b94adc09dacd-kube-api-access-mdqsx\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:16.943397 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.943314    2563 generic.go:356] "Generic (PLEG): container finished" podID="04d2e308-9a02-4a1b-aa34-b94adc09dacd" containerID="d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787" exitCode=0
Sep 09 07:19:16.943397 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.943378    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"
Sep 09 07:19:16.943613 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.943396    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz" event={"ID":"04d2e308-9a02-4a1b-aa34-b94adc09dacd","Type":"ContainerDied","Data":"d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787"}
Sep 09 07:19:16.943613 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.943475    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz" event={"ID":"04d2e308-9a02-4a1b-aa34-b94adc09dacd","Type":"ContainerDied","Data":"00cb612e7d02da73101a8ea5d5d7538abab821b4b68aaa0f025733d964e3924e"}
Sep 09 07:19:16.943613 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.943505    2563 scope.go:117] "RemoveContainer" containerID="d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787"
Sep 09 07:19:16.953469 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.953450    2563 scope.go:117] "RemoveContainer" containerID="d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787"
Sep 09 07:19:16.953754 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:19:16.953736    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787\": container with ID starting with d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787 not found: ID does not exist" containerID="d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787"
Sep 09 07:19:16.953820 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.953762    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787"} err="failed to get container status \"d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787\": rpc error: code = NotFound desc = could not find container \"d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787\": container with ID starting with d8741d413542c076076cb7b8081b0932a86911edfcd0ac1615bc57bfc33ee787 not found: ID does not exist"
Sep 09 07:19:16.971792 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.971763    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"]
Sep 09 07:19:16.995972 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:16.995950    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-ea0f1c-6496f866fc-s8jfz"]
Sep 09 07:19:17.046412 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:17.046350    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:18.117707 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:18.117667    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04d2e308-9a02-4a1b-aa34-b94adc09dacd" path="/var/lib/kubelet/pods/04d2e308-9a02-4a1b-aa34-b94adc09dacd/volumes"
Sep 09 07:19:23.419935 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:23.419898    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:23.530019 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:23.529980    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:23.686963 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:23.686925    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"]
Sep 09 07:19:23.687232 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:23.687206    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy" containerID="cri-o://826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f" gracePeriod=30
Sep 09 07:19:23.710483 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:23.710454    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:24.168421 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:24.168339    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:28.949865 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:28.949841    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:19:29.012453 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.012361    2563 generic.go:356] "Generic (PLEG): container finished" podID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerID="826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f" exitCode=0
Sep 09 07:19:29.012453 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.012415    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"
Sep 09 07:19:29.012453 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.012442    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" event={"ID":"5cb80169-0b6d-4d17-9a4d-d835558960f1","Type":"ContainerDied","Data":"826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f"}
Sep 09 07:19:29.012746 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.012494    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87" event={"ID":"5cb80169-0b6d-4d17-9a4d-d835558960f1","Type":"ContainerDied","Data":"e638cb5844ae17bba442e7c74da617cdc0293ea59e386b1e4f36247518c5feb8"}
Sep 09 07:19:29.012746 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.012518    2563 scope.go:117] "RemoveContainer" containerID="826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f"
Sep 09 07:19:29.023679 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.023647    2563 scope.go:117] "RemoveContainer" containerID="0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37"
Sep 09 07:19:29.031281 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031255    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8s8tn\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-kube-api-access-8s8tn\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.031412 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031326    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-certs\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.031412 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031351    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-data\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.031412 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031404    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb80169-0b6d-4d17-9a4d-d835558960f1-istiod-ca-cert\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.031737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031434    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-socket\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.031737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031471    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-credential-socket\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.031737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031502    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-podinfo\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.031737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031563    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-envoy\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.031737 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031620    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-token\") pod \"5cb80169-0b6d-4d17-9a4d-d835558960f1\" (UID: \"5cb80169-0b6d-4d17-9a4d-d835558960f1\") "
Sep 09 07:19:29.032000 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031787    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:19:29.032000 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031856    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:19:29.032000 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031932    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:19:29.032000 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.031944    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:19:29.032232 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032167    2563 scope.go:117] "RemoveContainer" containerID="826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f"
Sep 09 07:19:29.032232 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032209    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5cb80169-0b6d-4d17-9a4d-d835558960f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:19:29.032572 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032454    2563 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.032572 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032479    2563 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-data\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.032572 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032494    2563 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5cb80169-0b6d-4d17-9a4d-d835558960f1-istiod-ca-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.032572 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032507    2563 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-workload-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.032572 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:19:29.032510    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f\": container with ID starting with 826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f not found: ID does not exist" containerID="826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f"
Sep 09 07:19:29.032572 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032550    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f"} err="failed to get container status \"826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f\": rpc error: code = NotFound desc = could not find container \"826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f\": container with ID starting with 826a2ba5f78a7f42a0cb46f0869981286614e4e13ba2a351ee5485d962fec78f not found: ID does not exist"
Sep 09 07:19:29.032572 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032578    2563 scope.go:117] "RemoveContainer" containerID="0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37"
Sep 09 07:19:29.033073 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032521    2563 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-credential-socket\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.033073 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:19:29.032951    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37\": container with ID starting with 0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37 not found: ID does not exist" containerID="0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37"
Sep 09 07:19:29.033073 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.032985    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37"} err="failed to get container status \"0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37\": rpc error: code = NotFound desc = could not find container \"0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37\": container with ID starting with 0fd24744e6479e6c3525bc0f58436f461fb18b7e546f0bcfd2ce755752156c37 not found: ID does not exist"
Sep 09 07:19:29.033770 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.033748    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-kube-api-access-8s8tn" (OuterVolumeSpecName: "kube-api-access-8s8tn") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "kube-api-access-8s8tn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:19:29.034074 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.034055    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:19:29.034157 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.034121    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 09 07:19:29.034248 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.034231    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5cb80169-0b6d-4d17-9a4d-d835558960f1" (UID: "5cb80169-0b6d-4d17-9a4d-d835558960f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 09 07:19:29.133640 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.133572    2563 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-podinfo\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.133640 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.133631    2563 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-envoy\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.133640 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.133643    2563 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-istio-token\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.133870 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.133652    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8s8tn\" (UniqueName: \"kubernetes.io/projected/5cb80169-0b6d-4d17-9a4d-d835558960f1-kube-api-access-8s8tn\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:29.336105 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.336075    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"]
Sep 09 07:19:29.343181 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:29.343144    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-ea0f1c-openshift-default-64797bfdd8-s2z87"]
Sep 09 07:19:30.117252 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:30.117220    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" path="/var/lib/kubelet/pods/5cb80169-0b6d-4d17-9a4d-d835558960f1/volumes"
Sep 09 07:19:34.633631 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.633401    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"]
Sep 09 07:19:34.634094 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.633731    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl" podUID="42ae422c-195e-4464-b7b8-2dec55397344" containerName="maas-api" containerID="cri-o://51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e" gracePeriod=30
Sep 09 07:19:34.668773 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.668741    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:34.737034 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.737003    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:34.838478 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.838446    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:34.887073 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.887014    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:19:34.910908 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.910869    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:34.992101 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.992072    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qm2h2\" (UniqueName: \"kubernetes.io/projected/42ae422c-195e-4464-b7b8-2dec55397344-kube-api-access-qm2h2\") pod \"42ae422c-195e-4464-b7b8-2dec55397344\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") "
Sep 09 07:19:34.992101 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.992107    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ae422c-195e-4464-b7b8-2dec55397344-openshift-service-ca\") pod \"42ae422c-195e-4464-b7b8-2dec55397344\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") "
Sep 09 07:19:34.992359 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.992133    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-metrics-certs\") pod \"42ae422c-195e-4464-b7b8-2dec55397344\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") "
Sep 09 07:19:34.992359 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.992200    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-maas-api-tls\") pod \"42ae422c-195e-4464-b7b8-2dec55397344\" (UID: \"42ae422c-195e-4464-b7b8-2dec55397344\") "
Sep 09 07:19:34.992510 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.992481    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42ae422c-195e-4464-b7b8-2dec55397344-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "42ae422c-195e-4464-b7b8-2dec55397344" (UID: "42ae422c-195e-4464-b7b8-2dec55397344"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:19:34.994300 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.994275    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "42ae422c-195e-4464-b7b8-2dec55397344" (UID: "42ae422c-195e-4464-b7b8-2dec55397344"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:19:34.994427 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.994315    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "42ae422c-195e-4464-b7b8-2dec55397344" (UID: "42ae422c-195e-4464-b7b8-2dec55397344"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:19:34.994427 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:34.994343    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42ae422c-195e-4464-b7b8-2dec55397344-kube-api-access-qm2h2" (OuterVolumeSpecName: "kube-api-access-qm2h2") pod "42ae422c-195e-4464-b7b8-2dec55397344" (UID: "42ae422c-195e-4464-b7b8-2dec55397344"). InnerVolumeSpecName "kube-api-access-qm2h2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:19:35.046303 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.046271    2563 generic.go:356] "Generic (PLEG): container finished" podID="42ae422c-195e-4464-b7b8-2dec55397344" containerID="51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e" exitCode=0
Sep 09 07:19:35.046497 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.046333    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"
Sep 09 07:19:35.046497 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.046356    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl" event={"ID":"42ae422c-195e-4464-b7b8-2dec55397344","Type":"ContainerDied","Data":"51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e"}
Sep 09 07:19:35.046497 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.046412    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl" event={"ID":"42ae422c-195e-4464-b7b8-2dec55397344","Type":"ContainerDied","Data":"741fe602e00ef167364a355f94521fcfa188f7757d7ca0b6e6b2dd115c6c0ba2"}
Sep 09 07:19:35.046497 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.046434    2563 scope.go:117] "RemoveContainer" containerID="51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e"
Sep 09 07:19:35.055890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.055876    2563 scope.go:117] "RemoveContainer" containerID="51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e"
Sep 09 07:19:35.056156 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:19:35.056141    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e\": container with ID starting with 51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e not found: ID does not exist" containerID="51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e"
Sep 09 07:19:35.056196 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.056162    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e"} err="failed to get container status \"51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e\": rpc error: code = NotFound desc = could not find container \"51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e\": container with ID starting with 51f34afa933b18c15564fc69b39d8acdbca389b3e4d26b9b686e4bc420bd106e not found: ID does not exist"
Sep 09 07:19:35.068634 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.068606    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"]
Sep 09 07:19:35.073034 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.073012    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-988b24-67d78bd49c-xjmhl"]
Sep 09 07:19:35.093331 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.093313    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qm2h2\" (UniqueName: \"kubernetes.io/projected/42ae422c-195e-4464-b7b8-2dec55397344-kube-api-access-qm2h2\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:35.093411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.093334    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/42ae422c-195e-4464-b7b8-2dec55397344-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:35.093411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.093345    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:35.093411 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.093354    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/42ae422c-195e-4464-b7b8-2dec55397344-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:35.225927 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:35.225898    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:36.116805 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:36.116763    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42ae422c-195e-4464-b7b8-2dec55397344" path="/var/lib/kubelet/pods/42ae422c-195e-4464-b7b8-2dec55397344/volumes"
Sep 09 07:19:42.599876 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:42.599836    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:42.701084 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:42.701032    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:19:44.062413 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.062375    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8cc576b77-mmjqb"]
Sep 09 07:19:44.062890 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.062674    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8cc576b77-mmjqb" podUID="e008d526-69c5-4d6f-ad4d-d4b404d40ff8" containerName="manager" containerID="cri-o://34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797" gracePeriod=30
Sep 09 07:19:44.314661 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.314604    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:19:44.381523 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.381489    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpz2n\" (UniqueName: \"kubernetes.io/projected/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-kube-api-access-tpz2n\") pod \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") "
Sep 09 07:19:44.381728 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.381597    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-metrics-certs\") pod \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") "
Sep 09 07:19:44.381728 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.381658    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-webhook-cert\") pod \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\" (UID: \"e008d526-69c5-4d6f-ad4d-d4b404d40ff8\") "
Sep 09 07:19:44.383602 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.383557    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e008d526-69c5-4d6f-ad4d-d4b404d40ff8" (UID: "e008d526-69c5-4d6f-ad4d-d4b404d40ff8"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:19:44.383705 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.383627    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "e008d526-69c5-4d6f-ad4d-d4b404d40ff8" (UID: "e008d526-69c5-4d6f-ad4d-d4b404d40ff8"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:19:44.383802 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.383782    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-kube-api-access-tpz2n" (OuterVolumeSpecName: "kube-api-access-tpz2n") pod "e008d526-69c5-4d6f-ad4d-d4b404d40ff8" (UID: "e008d526-69c5-4d6f-ad4d-d4b404d40ff8"). InnerVolumeSpecName "kube-api-access-tpz2n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:19:44.483357 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.483319    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpz2n\" (UniqueName: \"kubernetes.io/projected/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-kube-api-access-tpz2n\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:44.483357 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.483349    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:44.483357 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:44.483359    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e008d526-69c5-4d6f-ad4d-d4b404d40ff8-webhook-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:19:45.098409 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.098371    2563 generic.go:356] "Generic (PLEG): container finished" podID="e008d526-69c5-4d6f-ad4d-d4b404d40ff8" containerID="34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797" exitCode=0
Sep 09 07:19:45.098842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.098415    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-mmjqb"
Sep 09 07:19:45.098842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.098463    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-mmjqb" event={"ID":"e008d526-69c5-4d6f-ad4d-d4b404d40ff8","Type":"ContainerDied","Data":"34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797"}
Sep 09 07:19:45.098842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.098515    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-mmjqb" event={"ID":"e008d526-69c5-4d6f-ad4d-d4b404d40ff8","Type":"ContainerDied","Data":"ddc70132753bf527403b473c55fe545518754c868dc8b47565746fa9676fcde4"}
Sep 09 07:19:45.098842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.098541    2563 scope.go:117] "RemoveContainer" containerID="34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797"
Sep 09 07:19:45.108564 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.108546    2563 scope.go:117] "RemoveContainer" containerID="34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797"
Sep 09 07:19:45.108876 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:19:45.108858    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797\": container with ID starting with 34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797 not found: ID does not exist" containerID="34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797"
Sep 09 07:19:45.108932 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.108912    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797"} err="failed to get container status \"34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797\": rpc error: code = NotFound desc = could not find container \"34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797\": container with ID starting with 34c98eb6b40b99c6c6961aebd9acc1755663a37fcf2a40d336385e59ab115797 not found: ID does not exist"
Sep 09 07:19:45.120206 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.120181    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8cc576b77-mmjqb"]
Sep 09 07:19:45.123371 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:45.123347    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8cc576b77-mmjqb"]
Sep 09 07:19:46.117476 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:19:46.117433    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e008d526-69c5-4d6f-ad4d-d4b404d40ff8" path="/var/lib/kubelet/pods/e008d526-69c5-4d6f-ad4d-d4b404d40ff8/volumes"
Sep 09 07:20:24.131006 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:24.130968    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:20:24.184746 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:24.184715    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:20:45.940645 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.940606    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8cc576b77-znxmk"]
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941000    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941011    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941019    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941024    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941033    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941038    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941062    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d17119e8-f9e6-4c24-8700-c0c72f64487a" containerName="maas-api"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941068    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="d17119e8-f9e6-4c24-8700-c0c72f64487a" containerName="maas-api"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941074    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04d2e308-9a02-4a1b-aa34-b94adc09dacd" containerName="maas-api"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941080    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="04d2e308-9a02-4a1b-aa34-b94adc09dacd" containerName="maas-api"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941097    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42ae422c-195e-4464-b7b8-2dec55397344" containerName="maas-api"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941102    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="42ae422c-195e-4464-b7b8-2dec55397344" containerName="maas-api"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941110    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy"
Sep 09 07:20:45.941111 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941115    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941120    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941126    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941131    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e008d526-69c5-4d6f-ad4d-d4b404d40ff8" containerName="manager"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941136    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="e008d526-69c5-4d6f-ad4d-d4b404d40ff8" containerName="manager"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941234    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941263    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941275    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="42ae422c-195e-4464-b7b8-2dec55397344" containerName="maas-api"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941289    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941322    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941334    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="e008d526-69c5-4d6f-ad4d-d4b404d40ff8" containerName="manager"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941344    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="40df5d19-ec78-4f40-a782-a7afad941d53" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941353    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="04d2e308-9a02-4a1b-aa34-b94adc09dacd" containerName="maas-api"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941368    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dea144c-972a-4ed7-a4b3-22d5db87659e" containerName="istio-proxy"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941374    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="d17119e8-f9e6-4c24-8700-c0c72f64487a" containerName="maas-api"
Sep 09 07:20:45.941747 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.941380    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="5cb80169-0b6d-4d17-9a4d-d835558960f1" containerName="istio-proxy"
Sep 09 07:20:45.944642 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.944625    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:45.947212 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.947190    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 07:20:45.948155 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.948135    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 07:20:45.948249 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.948160    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dmhkc\""
Sep 09 07:20:45.948249 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.948144    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 07:20:45.953931 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:45.953907    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8cc576b77-znxmk"]
Sep 09 07:20:46.045844 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.045795    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-webhook-cert\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.046020 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.045863    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-metrics-certs\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.046020 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.045888    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtq75\" (UniqueName: \"kubernetes.io/projected/28f6a84b-4625-4614-ac7e-3a2be1406bfd-kube-api-access-vtq75\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.146538 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.146485    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-webhook-cert\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.146741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.146574    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-metrics-certs\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.146741 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.146634    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtq75\" (UniqueName: \"kubernetes.io/projected/28f6a84b-4625-4614-ac7e-3a2be1406bfd-kube-api-access-vtq75\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.149052 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.149018    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-webhook-cert\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.149181 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.149052    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-metrics-certs\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.154939 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.154917    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtq75\" (UniqueName: \"kubernetes.io/projected/28f6a84b-4625-4614-ac7e-3a2be1406bfd-kube-api-access-vtq75\") pod \"maas-controller-8cc576b77-znxmk\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") " pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.255703 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.255583    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:46.527845 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.527818    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8cc576b77-znxmk"]
Sep 09 07:20:46.529439 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:20:46.529413    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28f6a84b_4625_4614_ac7e_3a2be1406bfd.slice/crio-23ed44096d8ec1e39fcab598712bedc8880c955bc63bd43d0134e4ce6d80cfa7 WatchSource:0}: Error finding container 23ed44096d8ec1e39fcab598712bedc8880c955bc63bd43d0134e4ce6d80cfa7: Status 404 returned error can't find the container with id 23ed44096d8ec1e39fcab598712bedc8880c955bc63bd43d0134e4ce6d80cfa7
Sep 09 07:20:46.530799 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:46.530780    2563 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 09 07:20:47.403147 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:47.403054    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-znxmk" event={"ID":"28f6a84b-4625-4614-ac7e-3a2be1406bfd","Type":"ContainerStarted","Data":"bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4"}
Sep 09 07:20:47.403560 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:47.403170    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:20:47.403560 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:47.403198    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-znxmk" event={"ID":"28f6a84b-4625-4614-ac7e-3a2be1406bfd","Type":"ContainerStarted","Data":"23ed44096d8ec1e39fcab598712bedc8880c955bc63bd43d0134e4ce6d80cfa7"}
Sep 09 07:20:47.420837 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:47.420790    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8cc576b77-znxmk" podStartSLOduration=1.938427146 podStartE2EDuration="2.42077345s" totalImagesPullingTime="482.346304ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:20:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:20:47.419959994 +0000 UTC m=+1779.790219085" watchObservedRunningTime="2026-09-09 07:20:47.42077345 +0000 UTC m=+1779.791032534"
Sep 09 07:20:58.410725 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:20:58.410691    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:21:06.704604 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:06.704559    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:21:07.701612 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:07.701555    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:21:07.794010 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:07.793977    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8cc576b77-znxmk"]
Sep 09 07:21:07.794527 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:07.794189    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-8cc576b77-znxmk" podUID="28f6a84b-4625-4614-ac7e-3a2be1406bfd" containerName="manager" containerID="cri-o://bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4" gracePeriod=30
Sep 09 07:21:08.046382 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.046356    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:21:08.138865 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.138837    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-metrics-certs\") pod \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") "
Sep 09 07:21:08.139024 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.138881    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtq75\" (UniqueName: \"kubernetes.io/projected/28f6a84b-4625-4614-ac7e-3a2be1406bfd-kube-api-access-vtq75\") pod \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") "
Sep 09 07:21:08.139024 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.138959    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-webhook-cert\") pod \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\" (UID: \"28f6a84b-4625-4614-ac7e-3a2be1406bfd\") "
Sep 09 07:21:08.140854 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.140825    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "28f6a84b-4625-4614-ac7e-3a2be1406bfd" (UID: "28f6a84b-4625-4614-ac7e-3a2be1406bfd"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:21:08.140984 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.140900    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28f6a84b-4625-4614-ac7e-3a2be1406bfd-kube-api-access-vtq75" (OuterVolumeSpecName: "kube-api-access-vtq75") pod "28f6a84b-4625-4614-ac7e-3a2be1406bfd" (UID: "28f6a84b-4625-4614-ac7e-3a2be1406bfd"). InnerVolumeSpecName "kube-api-access-vtq75". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:21:08.140984 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.140941    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "28f6a84b-4625-4614-ac7e-3a2be1406bfd" (UID: "28f6a84b-4625-4614-ac7e-3a2be1406bfd"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:21:08.239853 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.239775    2563 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-webhook-cert\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:21:08.239853 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.239799    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28f6a84b-4625-4614-ac7e-3a2be1406bfd-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:21:08.239853 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.239808    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtq75\" (UniqueName: \"kubernetes.io/projected/28f6a84b-4625-4614-ac7e-3a2be1406bfd-kube-api-access-vtq75\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:21:08.245557 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.245533    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:21:08.248926 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.248902    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:21:08.510643 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.510538    2563 generic.go:356] "Generic (PLEG): container finished" podID="28f6a84b-4625-4614-ac7e-3a2be1406bfd" containerID="bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4" exitCode=0
Sep 09 07:21:08.510643 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.510614    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-znxmk"
Sep 09 07:21:08.510842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.510615    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-znxmk" event={"ID":"28f6a84b-4625-4614-ac7e-3a2be1406bfd","Type":"ContainerDied","Data":"bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4"}
Sep 09 07:21:08.510842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.510729    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-znxmk" event={"ID":"28f6a84b-4625-4614-ac7e-3a2be1406bfd","Type":"ContainerDied","Data":"23ed44096d8ec1e39fcab598712bedc8880c955bc63bd43d0134e4ce6d80cfa7"}
Sep 09 07:21:08.510842 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.510750    2563 scope.go:117] "RemoveContainer" containerID="bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4"
Sep 09 07:21:08.519686 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.519663    2563 scope.go:117] "RemoveContainer" containerID="bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4"
Sep 09 07:21:08.519970 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:21:08.519952    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4\": container with ID starting with bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4 not found: ID does not exist" containerID="bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4"
Sep 09 07:21:08.520019 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.519978    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4"} err="failed to get container status \"bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4\": rpc error: code = NotFound desc = could not find container \"bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4\": container with ID starting with bea119ce7641519d1f24eb1b81aa899464593cb98a487b311783444e234393d4 not found: ID does not exist"
Sep 09 07:21:08.532950 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.532929    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-8cc576b77-znxmk"]
Sep 09 07:21:08.537103 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:08.537081    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-8cc576b77-znxmk"]
Sep 09 07:21:09.227492 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.227459    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:21:09.338430 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.338396    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-8cc576b77-hgnf4"]
Sep 09 07:21:09.338809 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.338797    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28f6a84b-4625-4614-ac7e-3a2be1406bfd" containerName="manager"
Sep 09 07:21:09.338858 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.338811    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="28f6a84b-4625-4614-ac7e-3a2be1406bfd" containerName="manager"
Sep 09 07:21:09.338858 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.338837    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy"
Sep 09 07:21:09.338858 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.338843    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="29ba2879-52db-4cc4-b25f-123c48f9556c" containerName="istio-proxy"
Sep 09 07:21:09.338953 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.338902    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="28f6a84b-4625-4614-ac7e-3a2be1406bfd" containerName="manager"
Sep 09 07:21:09.343217 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.343200    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.346686 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.346663    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-dmhkc\""
Sep 09 07:21:09.346821 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.346663    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 09 07:21:09.346821 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.346662    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 09 07:21:09.346821 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.346663    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 09 07:21:09.353003 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.352981    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8cc576b77-hgnf4"]
Sep 09 07:21:09.449526 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.449494    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c574f8e-210e-4222-be39-dffdb80ebcc5-webhook-cert\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.449705 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.449535    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b67c9\" (UniqueName: \"kubernetes.io/projected/2c574f8e-210e-4222-be39-dffdb80ebcc5-kube-api-access-b67c9\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.449705 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.449552    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c574f8e-210e-4222-be39-dffdb80ebcc5-metrics-certs\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.550936 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.550841    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c574f8e-210e-4222-be39-dffdb80ebcc5-webhook-cert\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.550936 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.550915    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b67c9\" (UniqueName: \"kubernetes.io/projected/2c574f8e-210e-4222-be39-dffdb80ebcc5-kube-api-access-b67c9\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.551168 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.550944    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c574f8e-210e-4222-be39-dffdb80ebcc5-metrics-certs\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.553384 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.553357    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2c574f8e-210e-4222-be39-dffdb80ebcc5-webhook-cert\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.553487 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.553385    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2c574f8e-210e-4222-be39-dffdb80ebcc5-metrics-certs\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.559607 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.559566    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b67c9\" (UniqueName: \"kubernetes.io/projected/2c574f8e-210e-4222-be39-dffdb80ebcc5-kube-api-access-b67c9\") pod \"maas-controller-8cc576b77-hgnf4\" (UID: \"2c574f8e-210e-4222-be39-dffdb80ebcc5\") " pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.653794 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.653760    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:09.928608 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:09.928562    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-8cc576b77-hgnf4"]
Sep 09 07:21:09.930633 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:21:09.930610    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c574f8e_210e_4222_be39_dffdb80ebcc5.slice/crio-02108ab64c71321b07085661a6cd546e8299fdb4f67c12b52078d2199dc9b21a WatchSource:0}: Error finding container 02108ab64c71321b07085661a6cd546e8299fdb4f67c12b52078d2199dc9b21a: Status 404 returned error can't find the container with id 02108ab64c71321b07085661a6cd546e8299fdb4f67c12b52078d2199dc9b21a
Sep 09 07:21:10.117577 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:10.117546    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28f6a84b-4625-4614-ac7e-3a2be1406bfd" path="/var/lib/kubelet/pods/28f6a84b-4625-4614-ac7e-3a2be1406bfd/volumes"
Sep 09 07:21:10.532453 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:10.532348    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-hgnf4" event={"ID":"2c574f8e-210e-4222-be39-dffdb80ebcc5","Type":"ContainerStarted","Data":"e47fcdb0719c60ceefb2707fd8b5e48818c791979a997e2e119dbe29b1acc529"}
Sep 09 07:21:10.532453 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:10.532428    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-8cc576b77-hgnf4" event={"ID":"2c574f8e-210e-4222-be39-dffdb80ebcc5","Type":"ContainerStarted","Data":"02108ab64c71321b07085661a6cd546e8299fdb4f67c12b52078d2199dc9b21a"}
Sep 09 07:21:10.533003 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:10.532457    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:21:10.551055 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:10.550998    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-8cc576b77-hgnf4" podStartSLOduration=1.208432423 podStartE2EDuration="1.550979609s" totalImagesPullingTime="342.547186ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:21:10.54988681 +0000 UTC m=+1802.920145898" watchObservedRunningTime="2026-09-09 07:21:10.550979609 +0000 UTC m=+1802.921238696"
Sep 09 07:21:14.445607 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:14.445553    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:21:21.540519 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:21:21.540482    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-8cc576b77-hgnf4"
Sep 09 07:22:48.100300 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:22:48.100248    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:23:18.431394 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:23:18.431352    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:24:13.263671 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:24:13.263573    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:24:16.649199 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:24:16.649155    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:24:44.347759 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:24:44.347718    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:24:45.155749 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:24:45.155717    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:25:02.037386 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.037353    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"]
Sep 09 07:25:02.041024 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.041007    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.052202 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.052170    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"]
Sep 09 07:25:02.120701 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.120667    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2g2mq\" (UniqueName: \"kubernetes.io/projected/8b17e891-9625-42d8-9244-9e3b65cc2631-kube-api-access-2g2mq\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.120701 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.120706    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b17e891-9625-42d8-9244-9e3b65cc2631-metrics-certs\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.120949 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.120738    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b17e891-9625-42d8-9244-9e3b65cc2631-openshift-service-ca\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.120949 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.120862    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b17e891-9625-42d8-9244-9e3b65cc2631-maas-api-tls\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.221822 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.221786    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2g2mq\" (UniqueName: \"kubernetes.io/projected/8b17e891-9625-42d8-9244-9e3b65cc2631-kube-api-access-2g2mq\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.221822 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.221828    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b17e891-9625-42d8-9244-9e3b65cc2631-metrics-certs\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.222031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.221886    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b17e891-9625-42d8-9244-9e3b65cc2631-openshift-service-ca\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.222031 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.221950    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b17e891-9625-42d8-9244-9e3b65cc2631-maas-api-tls\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.222738 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.222710    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8b17e891-9625-42d8-9244-9e3b65cc2631-openshift-service-ca\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.225106 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.225079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b17e891-9625-42d8-9244-9e3b65cc2631-metrics-certs\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.225224 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.225079    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8b17e891-9625-42d8-9244-9e3b65cc2631-maas-api-tls\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.230428 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.230403    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2g2mq\" (UniqueName: \"kubernetes.io/projected/8b17e891-9625-42d8-9244-9e3b65cc2631-kube-api-access-2g2mq\") pod \"maas-api-ff785c76b-6ct7f\" (UID: \"8b17e891-9625-42d8-9244-9e3b65cc2631\") " pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.354601 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.354506    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:02.627611 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.627569    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"]
Sep 09 07:25:02.629475 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:25:02.629449    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b17e891_9625_42d8_9244_9e3b65cc2631.slice/crio-ad09159147ee0c01d8b173a58fd21865c7e0690a03c187c16d700e11993ed2b0 WatchSource:0}: Error finding container ad09159147ee0c01d8b173a58fd21865c7e0690a03c187c16d700e11993ed2b0: Status 404 returned error can't find the container with id ad09159147ee0c01d8b173a58fd21865c7e0690a03c187c16d700e11993ed2b0
Sep 09 07:25:02.676103 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:02.676066    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f" event={"ID":"8b17e891-9625-42d8-9244-9e3b65cc2631","Type":"ContainerStarted","Data":"ad09159147ee0c01d8b173a58fd21865c7e0690a03c187c16d700e11993ed2b0"}
Sep 09 07:25:03.682781 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:03.682741    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f" event={"ID":"8b17e891-9625-42d8-9244-9e3b65cc2631","Type":"ContainerStarted","Data":"3b9eca31af72e93258c1496b595b150cddee42f402728854a5faabf1128d7ab2"}
Sep 09 07:25:03.683264 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:03.682873    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:03.702109 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:03.702063    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f" podStartSLOduration=1.270988038 podStartE2EDuration="1.702046068s" totalImagesPullingTime="431.05803ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-09 07:25:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:25:03.699699822 +0000 UTC m=+2036.069958909" watchObservedRunningTime="2026-09-09 07:25:03.702046068 +0000 UTC m=+2036.072305208"
Sep 09 07:25:09.693131 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:09.693102    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-ff785c76b-6ct7f"
Sep 09 07:25:09.741265 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:09.741235    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7998845878-sbdx4"]
Sep 09 07:25:09.741477 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:09.741432    2563 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4" podUID="c790ded1-63e5-4620-af5f-8274553be84a" containerName="maas-api" containerID="cri-o://43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7" gracePeriod=30
Sep 09 07:25:10.996294 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:10.996270    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:25:11.111392 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.111312    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ff9zw\" (UniqueName: \"kubernetes.io/projected/c790ded1-63e5-4620-af5f-8274553be84a-kube-api-access-ff9zw\") pod \"c790ded1-63e5-4620-af5f-8274553be84a\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") "
Sep 09 07:25:11.111392 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.111390    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-maas-api-tls\") pod \"c790ded1-63e5-4620-af5f-8274553be84a\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") "
Sep 09 07:25:11.111611 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.111489    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c790ded1-63e5-4620-af5f-8274553be84a-openshift-service-ca\") pod \"c790ded1-63e5-4620-af5f-8274553be84a\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") "
Sep 09 07:25:11.111611 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.111527    2563 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-metrics-certs\") pod \"c790ded1-63e5-4620-af5f-8274553be84a\" (UID: \"c790ded1-63e5-4620-af5f-8274553be84a\") "
Sep 09 07:25:11.111938 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.111905    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c790ded1-63e5-4620-af5f-8274553be84a-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c790ded1-63e5-4620-af5f-8274553be84a" (UID: "c790ded1-63e5-4620-af5f-8274553be84a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 09 07:25:11.113482 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.113459    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c790ded1-63e5-4620-af5f-8274553be84a" (UID: "c790ded1-63e5-4620-af5f-8274553be84a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:25:11.113609 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.113503    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "c790ded1-63e5-4620-af5f-8274553be84a" (UID: "c790ded1-63e5-4620-af5f-8274553be84a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 09 07:25:11.113609 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.113521    2563 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c790ded1-63e5-4620-af5f-8274553be84a-kube-api-access-ff9zw" (OuterVolumeSpecName: "kube-api-access-ff9zw") pod "c790ded1-63e5-4620-af5f-8274553be84a" (UID: "c790ded1-63e5-4620-af5f-8274553be84a"). InnerVolumeSpecName "kube-api-access-ff9zw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 09 07:25:11.213102 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.213061    2563 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff9zw\" (UniqueName: \"kubernetes.io/projected/c790ded1-63e5-4620-af5f-8274553be84a-kube-api-access-ff9zw\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:25:11.213102 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.213094    2563 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-maas-api-tls\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:25:11.213102 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.213107    2563 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c790ded1-63e5-4620-af5f-8274553be84a-openshift-service-ca\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:25:11.213338 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.213120    2563 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c790ded1-63e5-4620-af5f-8274553be84a-metrics-certs\") on node \"ip-10-0-128-206.ec2.internal\" DevicePath \"\""
Sep 09 07:25:11.730549 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.730517    2563 generic.go:356] "Generic (PLEG): container finished" podID="c790ded1-63e5-4620-af5f-8274553be84a" containerID="43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7" exitCode=0
Sep 09 07:25:11.730744 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.730560    2563 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4"
Sep 09 07:25:11.730744 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.730578    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4" event={"ID":"c790ded1-63e5-4620-af5f-8274553be84a","Type":"ContainerDied","Data":"43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7"}
Sep 09 07:25:11.730744 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.730634    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-7998845878-sbdx4" event={"ID":"c790ded1-63e5-4620-af5f-8274553be84a","Type":"ContainerDied","Data":"9d244e4d6bb50d0ee7f06ff66e4645574d22507baab2ff3a2932e56eadce6903"}
Sep 09 07:25:11.730744 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.730655    2563 scope.go:117] "RemoveContainer" containerID="43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7"
Sep 09 07:25:11.739763 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.739743    2563 scope.go:117] "RemoveContainer" containerID="43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7"
Sep 09 07:25:11.740034 ip-10-0-128-206 kubenswrapper[2563]: E0909 07:25:11.740016    2563 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7\": container with ID starting with 43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7 not found: ID does not exist" containerID="43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7"
Sep 09 07:25:11.740088 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.740044    2563 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7"} err="failed to get container status \"43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7\": rpc error: code = NotFound desc = could not find container \"43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7\": container with ID starting with 43a22e1c5dde7054573b0fdbbd5f30bc7808e5b72eca435eda8fc41ec33e08a7 not found: ID does not exist"
Sep 09 07:25:11.751955 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.751931    2563 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-7998845878-sbdx4"]
Sep 09 07:25:11.755739 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:11.755715    2563 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-7998845878-sbdx4"]
Sep 09 07:25:12.117447 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:12.117370    2563 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c790ded1-63e5-4620-af5f-8274553be84a" path="/var/lib/kubelet/pods/c790ded1-63e5-4620-af5f-8274553be84a/volumes"
Sep 09 07:25:28.471277 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:28.471239    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:25:28.528300 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:28.528262    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:25:37.592960 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:37.592876    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c5txd_25820297-c442-4e17-b230-faa402080612/manager/0.log"
Sep 09 07:25:37.857377 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:37.857284    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8cc576b77-hgnf4_2c574f8e-210e-4222-be39-dffdb80ebcc5/manager/0.log"
Sep 09 07:25:37.970941 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:37.970913    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-s66sj_c696554e-ecfa-48ae-8f88-d5e86f4f331c/server/0.log"
Sep 09 07:25:38.203054 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:38.203026    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c69cbcb-22cv5_7b72de59-473b-4f4a-997f-b5e950d1a3ef/manager/0.log"
Sep 09 07:25:38.208636 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:38.208612    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c69cbcb-22cv5_7b72de59-473b-4f4a-997f-b5e950d1a3ef/copy-manifests/0.log"
Sep 09 07:25:38.797242 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:38.797214    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-ff785c76b-6ct7f_8b17e891-9625-42d8-9244-9e3b65cc2631/maas-api/0.log"
Sep 09 07:25:39.161981 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:39.161871    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-wjtw5_c0ab5869-e46a-4e80-a924-982deffdc0a1/postgres/0.log"
Sep 09 07:25:40.074692 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:40.074659    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-2x89r_fa370629-32c7-49ff-bd39-fc91088bacee/manager/0.log"
Sep 09 07:25:40.184510 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:40.184484    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-7jp4c_9fdaf87c-e8a7-4481-9277-fe6a5dc58c53/manager/0.log"
Sep 09 07:25:41.121540 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:41.121510    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9v6l4_73a2873f-7174-4daf-8599-c935caa030c6/discovery/0.log"
Sep 09 07:25:41.351927 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:41.351880    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d6c696f5-z4nx7_0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c/kube-auth-proxy/0.log"
Sep 09 07:25:41.594152 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:41.594125    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-66x5x_328d013c-a98f-46aa-99ae-7edd3dfc8e9e/payload-pre-processing/0.log"
Sep 09 07:25:41.832975 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:41.832941    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58674b4548-wzsrc_e666318d-cdcd-483c-b61f-638bccc6cacf/router/0.log"
Sep 09 07:25:42.195553 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:42.195523    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-xk2mb_03c1f79d-5cbf-488d-b0bc-84cd525976fa/main/0.log"
Sep 09 07:25:42.311967 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:42.311941    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5bvqd_3c278d9b-2f37-4920-adfc-74ffb69a3e10/main/0.log"
Sep 09 07:25:42.774121 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:42.774089    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-5s44q_d42244bc-2de3-435a-8037-a5d77aa39b3f/main/0.log"
Sep 09 07:25:49.030898 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:49.030854    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:25:49.108826 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:49.108785    2563 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log" limit=1024
Sep 09 07:25:52.029340 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:52.029307    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7px44_857657b9-98d6-4f08-84cc-c832fa9d79ab/global-pull-secret-syncer/0.log"
Sep 09 07:25:52.165891 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:52.165859    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-97ml8_41f1fa06-a5b4-410b-8000-7b5ebbbf3a94/konnectivity-agent/0.log"
Sep 09 07:25:52.250312 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:52.250278    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-206.ec2.internal_05d7194320428eb238b4058701b24ef4/haproxy/0.log"
Sep 09 07:25:56.606770 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:56.606740    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-2x89r_fa370629-32c7-49ff-bd39-fc91088bacee/manager/0.log"
Sep 09 07:25:56.640245 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:56.640211    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-7jp4c_9fdaf87c-e8a7-4481-9277-fe6a5dc58c53/manager/0.log"
Sep 09 07:25:58.314968 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.314937    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62f67860-882f-4c2a-a6f1-1748815a0b21/alertmanager/0.log"
Sep 09 07:25:58.335762 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.335742    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62f67860-882f-4c2a-a6f1-1748815a0b21/config-reloader/0.log"
Sep 09 07:25:58.369264 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.369240    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62f67860-882f-4c2a-a6f1-1748815a0b21/kube-rbac-proxy-web/0.log"
Sep 09 07:25:58.389223 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.389198    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62f67860-882f-4c2a-a6f1-1748815a0b21/kube-rbac-proxy/0.log"
Sep 09 07:25:58.409418 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.409397    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62f67860-882f-4c2a-a6f1-1748815a0b21/kube-rbac-proxy-metric/0.log"
Sep 09 07:25:58.432981 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.432960    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62f67860-882f-4c2a-a6f1-1748815a0b21/prom-label-proxy/0.log"
Sep 09 07:25:58.455447 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.455425    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_62f67860-882f-4c2a-a6f1-1748815a0b21/init-config-reloader/0.log"
Sep 09 07:25:58.618049 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.617972    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-85bbbbbf4-t65fn_78be706c-fa6f-4c1a-b04b-874bd45d97c4/monitoring-plugin/0.log"
Sep 09 07:25:58.668529 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.668487    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hc4s4_5c06c26c-4ba8-4f7a-90b2-48eb2e25482c/node-exporter/0.log"
Sep 09 07:25:58.689496 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.689464    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hc4s4_5c06c26c-4ba8-4f7a-90b2-48eb2e25482c/kube-rbac-proxy/0.log"
Sep 09 07:25:58.708073 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:58.708047    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hc4s4_5c06c26c-4ba8-4f7a-90b2-48eb2e25482c/init-textfile/0.log"
Sep 09 07:25:59.248342 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.248302    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f9446bc69-cddks_2611e206-c6c9-4302-aa21-3ae4a70155d4/telemeter-client/0.log"
Sep 09 07:25:59.286971 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.286943    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f9446bc69-cddks_2611e206-c6c9-4302-aa21-3ae4a70155d4/reload/0.log"
Sep 09 07:25:59.311506 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.311468    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f9446bc69-cddks_2611e206-c6c9-4302-aa21-3ae4a70155d4/kube-rbac-proxy/0.log"
Sep 09 07:25:59.358135 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.358102    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545ffcc9b5-wzl6b_1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb/thanos-query/0.log"
Sep 09 07:25:59.382789 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.382758    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545ffcc9b5-wzl6b_1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb/kube-rbac-proxy-web/0.log"
Sep 09 07:25:59.407564 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.407528    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545ffcc9b5-wzl6b_1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb/kube-rbac-proxy/0.log"
Sep 09 07:25:59.438583 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.438549    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545ffcc9b5-wzl6b_1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb/prom-label-proxy/0.log"
Sep 09 07:25:59.464053 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.464029    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545ffcc9b5-wzl6b_1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb/kube-rbac-proxy-rules/0.log"
Sep 09 07:25:59.488950 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:25:59.488916    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-545ffcc9b5-wzl6b_1bf3d470-23ce-4b62-9e94-4e7b7ff70ddb/kube-rbac-proxy-metrics/0.log"
Sep 09 07:26:01.573713 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:01.573674    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:26:01.582620 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:01.582567    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/2.log"
Sep 09 07:26:02.095710 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.095679    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6767db5f76-9ddhv_85743ee0-2b6e-4cf2-b85e-2e65fa612b0d/console/0.log"
Sep 09 07:26:02.132039 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.132010    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-85d98c998f-rnpv4_f5bdc63f-58d6-400e-b84f-bbdb3e1dfbc2/download-server/0.log"
Sep 09 07:26:02.623230 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.623193    2563 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"]
Sep 09 07:26:02.623666 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.623650    2563 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c790ded1-63e5-4620-af5f-8274553be84a" containerName="maas-api"
Sep 09 07:26:02.623715 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.623667    2563 state_mem.go:107] "Deleted CPUSet assignment" podUID="c790ded1-63e5-4620-af5f-8274553be84a" containerName="maas-api"
Sep 09 07:26:02.623768 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.623744    2563 memory_manager.go:368] "RemoveStaleState removing state" podUID="c790ded1-63e5-4620-af5f-8274553be84a" containerName="maas-api"
Sep 09 07:26:02.627272 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.627249    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.630085 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.630058    2563 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-pv7q6\"/\"default-dockercfg-zmvt5\""
Sep 09 07:26:02.630234 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.630058    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pv7q6\"/\"kube-root-ca.crt\""
Sep 09 07:26:02.630520 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.630504    2563 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-pv7q6\"/\"openshift-service-ca.crt\""
Sep 09 07:26:02.639114 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.639091    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"]
Sep 09 07:26:02.689431 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.689391    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7pwt\" (UniqueName: \"kubernetes.io/projected/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-kube-api-access-d7pwt\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.689664 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.689441    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-podres\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.689664 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.689460    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-sys\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.689664 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.689546    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-lib-modules\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.689798 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.689673    2563 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-proc\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790084 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790041    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-proc\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790278 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790095    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7pwt\" (UniqueName: \"kubernetes.io/projected/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-kube-api-access-d7pwt\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790278 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790125    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-podres\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790278 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790141    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-sys\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790278 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790164    2563 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-lib-modules\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790278 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790195    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-proc\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790278 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790248    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-sys\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790496 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790298    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-lib-modules\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.790496 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.790303    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-podres\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.799315 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.799287    2563 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7pwt\" (UniqueName: \"kubernetes.io/projected/bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a-kube-api-access-d7pwt\") pod \"perf-node-gather-daemonset-j4szs\" (UID: \"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a\") " pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:02.937295 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:02.937260    2563 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:03.210390 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:03.210358    2563 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"]
Sep 09 07:26:03.212129 ip-10-0-128-206 kubenswrapper[2563]: W0909 07:26:03.212098    2563 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbba13f7e_ab7c_4ad5_aa38_fd0f38bc134a.slice/crio-9584bae55115924a021542faaa8174d01e8331c26f368db432e5c697b12eb171 WatchSource:0}: Error finding container 9584bae55115924a021542faaa8174d01e8331c26f368db432e5c697b12eb171: Status 404 returned error can't find the container with id 9584bae55115924a021542faaa8174d01e8331c26f368db432e5c697b12eb171
Sep 09 07:26:03.997097 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:03.997060    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs" event={"ID":"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a","Type":"ContainerStarted","Data":"e37d16f5d9aa376700a6a2f87f2d08b4af48f832e67e1fe6e77cad45248d68eb"}
Sep 09 07:26:03.997511 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:03.997112    2563 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs" event={"ID":"bba13f7e-ab7c-4ad5-aa38-fd0f38bc134a","Type":"ContainerStarted","Data":"9584bae55115924a021542faaa8174d01e8331c26f368db432e5c697b12eb171"}
Sep 09 07:26:03.997511 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:03.997162    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:04.014086 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:04.014036    2563 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs" podStartSLOduration=2.014021174 podStartE2EDuration="2.014021174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-09 07:26:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-09 07:26:04.012071615 +0000 UTC m=+2096.382330704" watchObservedRunningTime="2026-09-09 07:26:04.014021174 +0000 UTC m=+2096.384280262"
Sep 09 07:26:04.132745 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:04.132713    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8n858_9262d9d7-02ad-4160-876d-b6fc06627736/dns/0.log"
Sep 09 07:26:04.153761 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:04.153732    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8n858_9262d9d7-02ad-4160-876d-b6fc06627736/kube-rbac-proxy/0.log"
Sep 09 07:26:04.269907 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:04.269823    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rkmgz_11d5f98d-2bb0-4703-9b3a-2d72d8690e35/dns-node-resolver/0.log"
Sep 09 07:26:04.773336 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:04.773309    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8b947fcd8-zm48h_38060476-a2c7-4787-bcf1-39187f6f88d7/registry/0.log"
Sep 09 07:26:04.794628 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:04.794605    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-6r2sr_635184e6-443f-4df2-99e0-496ddc9fbe02/node-ca/0.log"
Sep 09 07:26:06.228959 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:06.228919    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9v6l4_73a2873f-7174-4daf-8599-c935caa030c6/discovery/0.log"
Sep 09 07:26:06.275376 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:06.275341    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d6c696f5-z4nx7_0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c/kube-auth-proxy/0.log"
Sep 09 07:26:06.360323 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:06.360283    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-66x5x_328d013c-a98f-46aa-99ae-7edd3dfc8e9e/payload-pre-processing/0.log"
Sep 09 07:26:06.418295 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:06.418268    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58674b4548-wzsrc_e666318d-cdcd-483c-b61f-638bccc6cacf/router/0.log"
Sep 09 07:26:07.328171 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:07.328139    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wncxl_d8ae03c8-633c-4fab-a8b3-6fb91116ee06/serve-healthcheck-canary/0.log"
Sep 09 07:26:07.993481 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:07.993442    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zlcml_6f96657d-2047-4456-94fb-5e5fa0d9e3db/kube-rbac-proxy/0.log"
Sep 09 07:26:08.014375 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:08.014343    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zlcml_6f96657d-2047-4456-94fb-5e5fa0d9e3db/exporter/0.log"
Sep 09 07:26:08.039414 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:08.039378    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zlcml_6f96657d-2047-4456-94fb-5e5fa0d9e3db/extractor/0.log"
Sep 09 07:26:08.277969 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:08.277888    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:26:08.281474 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:08.281451    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-85b454874b-zgbdk_b1881221-61f9-4b88-8296-96fc2078afa5/console-operator/1.log"
Sep 09 07:26:10.012055 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:10.012026    2563 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-pv7q6/perf-node-gather-daemonset-j4szs"
Sep 09 07:26:10.163899 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:10.163864    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-c5txd_25820297-c442-4e17-b230-faa402080612/manager/0.log"
Sep 09 07:26:10.448777 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:10.448746    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-8cc576b77-hgnf4_2c574f8e-210e-4222-be39-dffdb80ebcc5/manager/0.log"
Sep 09 07:26:10.467821 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:10.467795    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-s66sj_c696554e-ecfa-48ae-8f88-d5e86f4f331c/server/0.log"
Sep 09 07:26:10.587669 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:10.587640    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c69cbcb-22cv5_7b72de59-473b-4f4a-997f-b5e950d1a3ef/manager/0.log"
Sep 09 07:26:10.606869 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:10.606845    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7c69cbcb-22cv5_7b72de59-473b-4f4a-997f-b5e950d1a3ef/copy-manifests/0.log"
Sep 09 07:26:13.154932 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:13.154848    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9v6l4_73a2873f-7174-4daf-8599-c935caa030c6/discovery/0.log"
Sep 09 07:26:13.179220 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:13.179195    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d6c696f5-z4nx7_0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c/kube-auth-proxy/0.log"
Sep 09 07:26:13.242759 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:13.242727    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-66x5x_328d013c-a98f-46aa-99ae-7edd3dfc8e9e/payload-pre-processing/0.log"
Sep 09 07:26:13.288337 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:13.288306    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58674b4548-wzsrc_e666318d-cdcd-483c-b61f-638bccc6cacf/router/0.log"
Sep 09 07:26:15.304258 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:15.304223    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-9v6l4_73a2873f-7174-4daf-8599-c935caa030c6/discovery/0.log"
Sep 09 07:26:15.337333 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:15.337308    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-56d6c696f5-z4nx7_0cc63f11-fa25-45df-a2ba-8d7f2a1a0e5c/kube-auth-proxy/0.log"
Sep 09 07:26:15.414439 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:15.414402    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-59fbc47bf8-66x5x_328d013c-a98f-46aa-99ae-7edd3dfc8e9e/payload-pre-processing/0.log"
Sep 09 07:26:15.460055 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:15.460028    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-58674b4548-wzsrc_e666318d-cdcd-483c-b61f-638bccc6cacf/router/0.log"
Sep 09 07:26:17.418086 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:17.418056    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-qspv4_26903f73-be59-4b3a-9052-6218bddeacb2/migrator/0.log"
Sep 09 07:26:17.440539 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:17.440511    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-fdbb8fdff-qspv4_26903f73-be59-4b3a-9052-6218bddeacb2/graceful-termination/0.log"
Sep 09 07:26:17.782549 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:17.782516    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-pfbvh_2a5575d9-f5a0-4de3-9833-f30eed7135cb/kube-storage-version-migrator-operator/1.log"
Sep 09 07:26:17.784424 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:17.784396    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-8d7dd4675-pfbvh_2a5575d9-f5a0-4de3-9833-f30eed7135cb/kube-storage-version-migrator-operator/0.log"
Sep 09 07:26:18.756525 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:18.756494    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-7dhhp_7d928919-0da3-4cd5-88f6-d4227323bc6a/kube-multus/0.log"
Sep 09 07:26:19.088547 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.088470    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzxzm_4b30aa0c-9473-4ad8-a676-e12441e128cf/kube-multus-additional-cni-plugins/0.log"
Sep 09 07:26:19.108862 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.108833    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzxzm_4b30aa0c-9473-4ad8-a676-e12441e128cf/egress-router-binary-copy/0.log"
Sep 09 07:26:19.128386 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.128368    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzxzm_4b30aa0c-9473-4ad8-a676-e12441e128cf/cni-plugins/0.log"
Sep 09 07:26:19.146974 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.146953    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzxzm_4b30aa0c-9473-4ad8-a676-e12441e128cf/bond-cni-plugin/0.log"
Sep 09 07:26:19.165315 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.165294    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzxzm_4b30aa0c-9473-4ad8-a676-e12441e128cf/routeoverride-cni/0.log"
Sep 09 07:26:19.186484 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.186464    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzxzm_4b30aa0c-9473-4ad8-a676-e12441e128cf/whereabouts-cni-bincopy/0.log"
Sep 09 07:26:19.206373 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.206330    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-kzxzm_4b30aa0c-9473-4ad8-a676-e12441e128cf/whereabouts-cni/0.log"
Sep 09 07:26:19.325685 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.325642    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dkrzk_c5585130-4596-4fb1-8751-0dd1a062dc2e/network-metrics-daemon/0.log"
Sep 09 07:26:19.340698 ip-10-0-128-206 kubenswrapper[2563]: I0909 07:26:19.340628    2563 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-dkrzk_c5585130-4596-4fb1-8751-0dd1a062dc2e/kube-rbac-proxy/0.log"