Sep 10 14:19:54.118094 ip-10-0-142-68 systemd[1]: Starting Kubernetes Kubelet...
Sep 10 14:19:54.494974 ip-10-0-142-68 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:19:54.494974 ip-10-0-142-68 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 10 14:19:54.494974 ip-10-0-142-68 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:19:54.494974 ip-10-0-142-68 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 10 14:19:54.494974 ip-10-0-142-68 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 10 14:19:54.495666 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.494965    2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 10 14:19:54.498312 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498294    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:19:54.498312 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498312    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498315    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498320    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498323    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498326    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498329    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498332    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498334    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498337    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498340    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498343    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498345    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498349    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498352    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498355    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498357    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498360    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498362    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498365    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498367    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:19:54.498376 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498370    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498372    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498375    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498377    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498380    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498383    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498386    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498389    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498391    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498394    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498396    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498399    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498401    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498404    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498406    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498409    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498411    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498415    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498419    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:19:54.498903 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498421    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498424    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498427    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498429    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498431    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498434    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498437    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498440    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498442    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498445    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498447    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498449    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498452    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498454    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498457    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498460    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498463    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498465    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498468    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498470    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:19:54.499436 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498473    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498475    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498478    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498480    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498483    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498486    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498488    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498491    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498493    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498496    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498498    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498501    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498503    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498506    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498509    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498511    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498513    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498516    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498518    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498521    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:19:54.499972 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498524    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498526    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498529    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498531    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498533    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498536    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498538    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498544    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498547    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498549    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498552    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498554    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498558    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498561    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498564    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498568    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498571    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498574    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.498577    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:19:54.500686 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499340    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499347    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499350    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499353    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499356    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499358    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499361    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499363    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499366    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499368    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499371    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499373    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499376    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499384    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499386    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499389    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499391    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499394    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499396    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499399    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:19:54.501250 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499402    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499405    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499408    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499410    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499412    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499416    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499419    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499423    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499426    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499428    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499431    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499433    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499436    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499438    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499441    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499443    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499446    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499448    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499451    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499453    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:19:54.501878 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499456    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499459    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499461    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499464    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499466    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499468    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499478    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499481    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499483    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499485    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499488    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499490    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499493    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499496    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499499    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499501    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499504    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499507    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499511    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:19:54.502365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499515    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499518    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499521    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499524    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499526    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499529    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499532    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499535    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499537    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499540    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499543    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499545    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499548    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499551    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499553    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499556    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499558    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499561    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499564    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499566    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:19:54.502837 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499574    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499577    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499579    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499582    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499584    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499587    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499589    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499592    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499595    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499598    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499600    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499603    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499605    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499608    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499611    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499613    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499616    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499618    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499621    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:19:54.503322 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.499624    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501903    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501914    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501919    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501922    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501926    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501929    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501932    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501935    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501938    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501940    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501943    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501945    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501948    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501951    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501954    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501956    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501959    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501962    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501966    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:19:54.503797 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501970    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501974    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501978    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501981    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501984    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501986    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501989    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501991    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501993    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.501999    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502002    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502005    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502007    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502010    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502013    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502015    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502018    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502020    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502023    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502026    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:19:54.504272 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502029    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502031    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502034    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502036    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502038    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502041    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502046    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502050    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502054    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502059    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502062    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502064    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502067    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502070    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502072    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502074    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502077    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502082    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502085    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502087    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:19:54.504804 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502090    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502093    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502095    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502097    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502100    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502103    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502105    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502108    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502111    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502113    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502116    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502118    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502121    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502126    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502130    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502134    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502138    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502143    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502147    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:19:54.505294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502150    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502152    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502155    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502158    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502160    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502164    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502166    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502169    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502173    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502176    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502181    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502184    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502187    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502189    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502193    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502197    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502200    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502204    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502208    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:19:54.505759 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502212    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.502216    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502294    2578 flags.go:64] FLAG: --address="0.0.0.0"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502332    2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502339    2578 flags.go:64] FLAG: --anonymous-auth="true"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502344    2578 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502347    2578 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502350    2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502354    2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502358    2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502361    2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502365    2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502372    2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502377    2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502382    2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502385    2578 flags.go:64] FLAG: --cgroup-root=""
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502388    2578 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502391    2578 flags.go:64] FLAG: --client-ca-file=""
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502393    2578 flags.go:64] FLAG: --cloud-provider="external"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502397    2578 flags.go:64] FLAG: --cluster-dns="[]"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502402    2578 flags.go:64] FLAG: --cluster-domain=""
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502405    2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502408    2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502411    2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 10 14:19:54.506240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502414    2578 flags.go:64] FLAG: --container-log-max-files="5"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502418    2578 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502421    2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502424    2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502427    2578 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502430    2578 flags.go:64] FLAG: --contention-profiling="false"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502433    2578 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502436    2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502439    2578 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502442    2578 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502451    2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502456    2578 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502461    2578 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502464    2578 flags.go:64] FLAG: --enable-load-reader="false"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502467    2578 flags.go:64] FLAG: --enable-server="true"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502470    2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502474    2578 flags.go:64] FLAG: --event-burst="100"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502477    2578 flags.go:64] FLAG: --event-qps="50"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502479    2578 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502482    2578 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502485    2578 flags.go:64] FLAG: --eviction-hard=""
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502489    2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502492    2578 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502494    2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502497    2578 flags.go:64] FLAG: --eviction-soft=""
Sep 10 14:19:54.506799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502500    2578 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502503    2578 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502506    2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502509    2578 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502512    2578 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502516    2578 flags.go:64] FLAG: --fail-swap-on="true"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502518    2578 flags.go:64] FLAG: --feature-gates=""
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502522    2578 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502526    2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502531    2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502536    2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502541    2578 flags.go:64] FLAG: --healthz-port="10248"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502544    2578 flags.go:64] FLAG: --help="false"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502547    2578 flags.go:64] FLAG: --hostname-override="ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502550    2578 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502553    2578 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502556    2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502559    2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502562    2578 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502565    2578 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502567    2578 flags.go:64] FLAG: --image-service-endpoint=""
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502570    2578 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502573    2578 flags.go:64] FLAG: --kube-api-burst="100"
Sep 10 14:19:54.507395 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502576    2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502578    2578 flags.go:64] FLAG: --kube-api-qps="50"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502581    2578 flags.go:64] FLAG: --kube-reserved=""
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502584    2578 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502586    2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502589    2578 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502591    2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502594    2578 flags.go:64] FLAG: --lock-file=""
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502597    2578 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502599    2578 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502602    2578 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502610    2578 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502614    2578 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502620    2578 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502623    2578 flags.go:64] FLAG: --logging-format="text"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502626    2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502629    2578 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502632    2578 flags.go:64] FLAG: --manifest-url=""
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502634    2578 flags.go:64] FLAG: --manifest-url-header=""
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502638    2578 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502641    2578 flags.go:64] FLAG: --max-open-files="1000000"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502644    2578 flags.go:64] FLAG: --max-pods="110"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502647    2578 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502650    2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502653    2578 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 10 14:19:54.507981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502655    2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502658    2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502661    2578 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502664    2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502671    2578 flags.go:64] FLAG: --node-status-max-images="50"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502674    2578 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502676    2578 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502679    2578 flags.go:64] FLAG: --pod-cidr=""
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502682    2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8ccbacc999dbf9abad8f4009240188312b683f36ced4763eed166c1500c164b1"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502689    2578 flags.go:64] FLAG: --pod-manifest-path=""
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502693    2578 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502698    2578 flags.go:64] FLAG: --pods-per-core="0"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502702    2578 flags.go:64] FLAG: --port="10250"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502705    2578 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502708    2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e626ce444cfad047"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502711    2578 flags.go:64] FLAG: --qos-reserved=""
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502714    2578 flags.go:64] FLAG: --read-only-port="10255"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502717    2578 flags.go:64] FLAG: --register-node="true"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502720    2578 flags.go:64] FLAG: --register-with-taints=""
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502723    2578 flags.go:64] FLAG: --registry-burst="10"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502726    2578 flags.go:64] FLAG: --registry-qps="5"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502731    2578 flags.go:64] FLAG: --reserved-cpus=""
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502733    2578 flags.go:64] FLAG: --reserved-memory=""
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502737    2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502740    2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 10 14:19:54.508584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502756    2578 flags.go:64] FLAG: --rotate-certificates="false"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502760    2578 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502762    2578 flags.go:64] FLAG: --runonce="false"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502767    2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502771    2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502776    2578 flags.go:64] FLAG: --seccomp-default="false"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502781    2578 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502784    2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502787    2578 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502790    2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502792    2578 flags.go:64] FLAG: --storage-driver-password="root"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502795    2578 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502798    2578 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502800    2578 flags.go:64] FLAG: --storage-driver-user="root"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502803    2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502806    2578 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502808    2578 flags.go:64] FLAG: --system-cgroups=""
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502811    2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502816    2578 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502819    2578 flags.go:64] FLAG: --tls-cert-file=""
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502821    2578 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502826    2578 flags.go:64] FLAG: --tls-min-version=""
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502828    2578 flags.go:64] FLAG: --tls-private-key-file=""
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502831    2578 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502834    2578 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 10 14:19:54.509283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502837    2578 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502840    2578 flags.go:64] FLAG: --v="2"
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502845    2578 flags.go:64] FLAG: --version="false"
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502852    2578 flags.go:64] FLAG: --vmodule=""
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502864    2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.502869    2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503059    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503065    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503070    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503074    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503077    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503080    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503082    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503087    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503091    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503095    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503100    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503104    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503107    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503110    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:19:54.510362 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503112    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503115    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503117    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503120    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503122    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503125    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503127    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503129    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503132    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503134    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503137    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503139    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503142    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503144    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503147    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503151    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503153    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503158    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503161    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503163    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:19:54.511228 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503165    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503169    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503173    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503179    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503184    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503187    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503190    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503193    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503196    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503198    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503201    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503204    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503206    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503209    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503212    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503215    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503217    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503220    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503223    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:19:54.511889 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503225    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503227    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503230    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503232    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503235    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503237    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503239    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503242    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503246    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503249    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503255    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503261    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503264    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503267    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503269    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503271    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503274    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503276    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503279    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503281    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:19:54.512457 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503284    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503286    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503289    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503291    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503294    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503296    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503298    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503301    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503304    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503306    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503309    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503311    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503313    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503316    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503318    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503321    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503323    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503326    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503331    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503335    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:19:54.513337 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503341    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503345    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503348    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503351    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503354    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.503357    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.503845    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.511523    2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.511541    2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.512286    2578 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.512298    2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512440    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512448    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512453    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512458    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:19:54.514105 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512462    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512466    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512470    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512475    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512478    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512483    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512487    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512491    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512496    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512500    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512505    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512509    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512513    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512517    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512522    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512526    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512530    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512534    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512539    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512542    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:19:54.514610 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512547    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512552    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512556    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512560    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512564    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512569    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512574    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512580    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512586    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512591    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512595    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512599    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512603    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512607    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512610    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512614    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512619    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512623    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512627    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:19:54.515475 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512631    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512635    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512639    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512646    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512651    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512656    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512660    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512665    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512670    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512674    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512679    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512683    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512687    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512691    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512696    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512701    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512705    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512710    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512715    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:19:54.516208 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512719    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512723    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512727    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512731    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512735    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512739    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512763    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512768    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512772    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512776    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512780    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512783    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512787    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512791    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512797    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512802    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512807    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512813    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512817    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512821    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:19:54.516665 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512826    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512830    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512834    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512838    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512842    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512846    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512850    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512855    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512859    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512864    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512868    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512873    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512877    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512880    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512884    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512888    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:19:54.517207 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.512893    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.512900    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513117    2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513125    2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513131    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513138    2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513144    2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513150    2578 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513155    2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513160    2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513164    2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513168    2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513172    2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513177    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 10 14:19:54.517588 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513181    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513185    2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513189    2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513193    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513197    2578 feature_gate.go:328] unrecognized feature gate: Example
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513201    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513205    2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513208    2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513212    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513216    2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513220    2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513224    2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513230    2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513234    2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513238    2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513242    2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513246    2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513250    2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513256    2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513261    2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 10 14:19:54.518080 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513266    2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513270    2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513275    2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513279    2578 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513283    2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513287    2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513290    2578 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513294    2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513298    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513302    2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513306    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513310    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513314    2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513318    2578 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513322    2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513326    2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513330    2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513334    2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513338    2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513341    2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 10 14:19:54.518563 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513345    2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513348    2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513352    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513356    2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513360    2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513364    2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513368    2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513372    2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513376    2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513379    2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513383    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513387    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513391    2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513395    2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513399    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513403    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513407    2578 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513411    2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513415    2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513419    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 10 14:19:54.519077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513423    2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513427    2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513431    2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513435    2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513439    2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513443    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513446    2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513450    2578 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513455    2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513461    2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513465    2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513469    2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513473    2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513476    2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513480    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513484    2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513488    2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513492    2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513498    2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513502    2578 feature_gate.go:328] unrecognized feature gate: Example2
Sep 10 14:19:54.519933 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513506    2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513509    2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513513    2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513518    2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513522    2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513525    2578 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:54.513529    2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.513537    2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.513684    2578 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.515863    2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.516943    2578 server.go:1019] "Starting client certificate rotation"
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.517043    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.517107    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 10 14:19:54.520405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.519826    2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:19:54.522179 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.522151    2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 10 14:19:54.532724 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.532704    2578 log.go:25] "Validated CRI v1 runtime API"
Sep 10 14:19:54.536508 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.536489    2578 log.go:25] "Validated CRI v1 image API"
Sep 10 14:19:54.544760 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.544723    2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 10 14:19:54.548170 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.548150    2578 fs.go:135] Filesystem UUIDs: map[5d0e7d96-86cf-4cd3-a94d-7426d71fb147:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 9eecfe9c-5030-49b6-9cbb-bb0f28825efa:/dev/nvme0n1p4]
Sep 10 14:19:54.548249 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.548170    2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 10 14:19:54.549014 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.548998    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:19:54.553154 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.553045    2578 manager.go:216] Machine: {Timestamp:2026-09-10 14:19:54.552550637 +0000 UTC m=+0.320273412 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3219860 MemoryCapacity:32812163072 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec21227447d148d7720d45e4ab805c96 SystemUUID:ec212274-47d1-48d7-720d-45e4ab805c96 BootID:8ab61d42-2f57-4c61-afdc-07346c707800 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406081536 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a8:5e:50:20:99 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a8:5e:50:20:99 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:a8:c5:1f:d8:fe Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812163072 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 10 14:19:54.553154 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.553149    2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 10 14:19:54.553272 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.553233    2578 manager.go:232] Version: {KernelVersion:5.14.0-570.138.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260901-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 10 14:19:54.554271 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.554248    2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 10 14:19:54.554410 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.554274    2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-68.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 10 14:19:54.554970 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.554958    2578 topology_manager.go:138] "Creating topology manager with none policy"
Sep 10 14:19:54.555001 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.554973    2578 container_manager_linux.go:309] "Creating device plugin manager"
Sep 10 14:19:54.555001 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.554980    2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:19:54.555680 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.555670    2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 10 14:19:54.557143 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.557127    2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 10 14:19:54.561665 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.561651    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:19:54.561807 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.561797    2578 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 10 14:19:54.561848 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.561828    2578 kubelet.go:486] "Attempting to sync node with API server"
Sep 10 14:19:54.561848 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.561838    2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 10 14:19:54.561934 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.561858    2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 10 14:19:54.561934 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.561870    2578 kubelet.go:398] "Adding apiserver pod source"
Sep 10 14:19:54.561934 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.561879    2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 10 14:19:54.562309 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.562298    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:19:54.562341 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.562321    2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 10 14:19:54.564830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.564815    2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 10 14:19:54.567196 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567169    2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 10 14:19:54.567298 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567214    2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 10 14:19:54.567910 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567885    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567910    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567923    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567931    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567943    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567950    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567967    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567974    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567983    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 10 14:19:54.567987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.567992    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 10 14:19:54.568231 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.568013    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 10 14:19:54.568231 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.568019    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 10 14:19:54.568568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.568556    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 10 14:19:54.568597 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.568572    2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 10 14:19:54.570399 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.570378    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wh9gf"
Sep 10 14:19:54.570888 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.570868    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 10 14:19:54.571058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.571036    2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-68.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 10 14:19:54.571058 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.571046    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 10 14:19:54.572508 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.572492    2578 server.go:1268] "Started kubelet"
Sep 10 14:19:54.572596 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.572543    2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 10 14:19:54.572910 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.572656    2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 10 14:19:54.573097 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.573075    2578 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 10 14:19:54.573452 ip-10-0-142-68 systemd[1]: Started Kubernetes Kubelet.
Sep 10 14:19:54.574474 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.574417    2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 10 14:19:54.575635 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.575602    2578 server.go:310] "Adding debug handlers to kubelet server"
Sep 10 14:19:54.576816 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.576739    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-wh9gf"
Sep 10 14:19:54.577488 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.577464    2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 10 14:19:54.580304 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.580283    2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 10 14:19:54.580400 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.580304    2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 10 14:19:54.580400 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.580372    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:54.580897 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.580878    2578 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 10 14:19:54.580897 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.580898    2578 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 10 14:19:54.581039 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.580878    2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 10 14:19:54.581039 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.580976    2578 reconstruct.go:97] "Volume reconstruction finished"
Sep 10 14:19:54.581039 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.580984    2578 reconciler.go:29] "Reconciler: start to sync state"
Sep 10 14:19:54.581247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.581232    2578 factory.go:55] Registering systemd factory
Sep 10 14:19:54.581298 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.581252    2578 factory.go:223] Registration of the systemd container factory successfully
Sep 10 14:19:54.581601 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.581584    2578 factory.go:153] Registering CRI-O factory
Sep 10 14:19:54.581674 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.581605    2578 factory.go:223] Registration of the crio container factory successfully
Sep 10 14:19:54.581726 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.581673    2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 10 14:19:54.581726 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.581694    2578 factory.go:103] Registering Raw factory
Sep 10 14:19:54.581726 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.581710    2578 manager.go:1196] Started watching for new ooms in manager
Sep 10 14:19:54.582134 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.582109    2578 manager.go:349] Starting recovery of all containers
Sep 10 14:19:54.584341 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.584055    2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-68.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 10 14:19:54.584341 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.584201    2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 10 14:19:54.586869 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.584274    2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-68.ec2.internal.18d3fb7dc3e5388a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-68.ec2.internal,UID:ip-10-0-142-68.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-68.ec2.internal,},FirstTimestamp:2026-09-10 14:19:54.572281994 +0000 UTC m=+0.340004765,LastTimestamp:2026-09-10 14:19:54.572281994 +0000 UTC m=+0.340004765,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-68.ec2.internal,}"
Sep 10 14:19:54.594589 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.594574    2578 manager.go:354] Recovery completed
Sep 10 14:19:54.596397 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.596379    2578 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
Sep 10 14:19:54.599554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.599542    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:19:54.602063 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.602045    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:19:54.602122 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.602074    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:19:54.602122 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.602084    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:19:54.602767 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.602735    2578 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 10 14:19:54.602767 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.602766    2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 10 14:19:54.602894 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.602772    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:19:54.602894 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.602784    2578 state_mem.go:36] "Initialized new in-memory state store"
Sep 10 14:19:54.603590 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.603575    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:19:54.603657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.603603    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:19:54.603657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.603630    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:19:54.604946 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.604932    2578 policy_none.go:49] "None policy: Start"
Sep 10 14:19:54.605001 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.604959    2578 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 10 14:19:54.605001 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.604973    2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 10 14:19:54.607056 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.607045    2578 policy_none.go:47] "Start"
Sep 10 14:19:54.639529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.639334    2578 manager.go:337] "Starting Device Plugin manager"
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.639562    2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.639576    2578 server.go:87] "Starting device plugin registration server"
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.639836    2578 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.639846    2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.639966    2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.640072    2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.640094    2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.640587    2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 10 14:19:54.663337 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.640636    2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:54.716074 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.716038    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 10 14:19:54.717393 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.717376    2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 10 14:19:54.717498 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.717426    2578 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 10 14:19:54.717498 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.717455    2578 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 10 14:19:54.717580 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.717498    2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 10 14:19:54.720094 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.720075    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:19:54.740293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.740272    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:19:54.741147 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.741129    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:19:54.741229 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.741160    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:19:54.741229 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.741170    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:19:54.741229 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.741192    2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.751492 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.751447    2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.751492 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.751471    2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-68.ec2.internal\": node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:54.766024 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.766006    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:54.817620 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.817583    2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal"]
Sep 10 14:19:54.817837 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.817699    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:19:54.819250 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.819233    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:19:54.819338 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.819262    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:19:54.819338 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.819272    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:19:54.821532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.821520    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:19:54.821658 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.821638    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.821707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.821665    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:19:54.822285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.822267    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:19:54.822285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.822278    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:19:54.822285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.822288    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:19:54.822455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.822298    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:19:54.822455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.822298    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:19:54.822455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.822405    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:19:54.824504 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.824490    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.824567 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.824516    2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 10 14:19:54.825175 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.825149    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientMemory"
Sep 10 14:19:54.825256 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.825178    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasNoDiskPressure"
Sep 10 14:19:54.825256 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.825190    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeHasSufficientPID"
Sep 10 14:19:54.850446 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.850427    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-68.ec2.internal\" not found" node="ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.854799 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.854782    2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-68.ec2.internal\" not found" node="ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.866842 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.866822    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:54.882925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.882896    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49f521e0eb96ffa477f62f278bc4f4c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal\" (UID: \"49f521e0eb96ffa477f62f278bc4f4c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.882925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.882933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49f521e0eb96ffa477f62f278bc4f4c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal\" (UID: \"49f521e0eb96ffa477f62f278bc4f4c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.883062 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.882952    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/579903845495bf197e8392d11f220146-config\") pod \"kube-apiserver-proxy-ip-10-0-142-68.ec2.internal\" (UID: \"579903845495bf197e8392d11f220146\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.967874 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:54.967834    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:54.983207 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.983182    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49f521e0eb96ffa477f62f278bc4f4c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal\" (UID: \"49f521e0eb96ffa477f62f278bc4f4c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.983318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.983216    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/579903845495bf197e8392d11f220146-config\") pod \"kube-apiserver-proxy-ip-10-0-142-68.ec2.internal\" (UID: \"579903845495bf197e8392d11f220146\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.983318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.983245    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49f521e0eb96ffa477f62f278bc4f4c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal\" (UID: \"49f521e0eb96ffa477f62f278bc4f4c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.983318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.983293    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/49f521e0eb96ffa477f62f278bc4f4c4-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal\" (UID: \"49f521e0eb96ffa477f62f278bc4f4c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.983419 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.983299    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/49f521e0eb96ffa477f62f278bc4f4c4-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal\" (UID: \"49f521e0eb96ffa477f62f278bc4f4c4\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:54.983419 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:54.983298    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/579903845495bf197e8392d11f220146-config\") pod \"kube-apiserver-proxy-ip-10-0-142-68.ec2.internal\" (UID: \"579903845495bf197e8392d11f220146\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:55.068730 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:55.068665    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:55.152252 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.152217    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:55.158182 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.158164    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:55.169809 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:55.169784    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:55.270385 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:55.270345    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:55.370934 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:55.370859    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:55.471498 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:55.471463    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:55.517030 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.517007    2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 10 14:19:55.517424 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.517158    2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 10 14:19:55.571550 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:55.571521    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:55.583238 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.583207    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-09 14:14:55 +0000 UTC" deadline="2028-02-21 13:14:56.71227658 +0000 UTC"
Sep 10 14:19:55.583238 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.583233    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12694h55m1.129046351s"
Sep 10 14:19:55.672168 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:55.672102    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:55.773119 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:55.773083    2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-68.ec2.internal\" not found"
Sep 10 14:19:55.846669 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.846643    2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:19:55.852660 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.852640    2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:19:55.876478 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:55.876445    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod579903845495bf197e8392d11f220146.slice/crio-562673dd0871b1ea1a75b73a16ba941e8694241acd596daee42f8f82b2a4ba6e WatchSource:0}: Error finding container 562673dd0871b1ea1a75b73a16ba941e8694241acd596daee42f8f82b2a4ba6e: Status 404 returned error can't find the container with id 562673dd0871b1ea1a75b73a16ba941e8694241acd596daee42f8f82b2a4ba6e
Sep 10 14:19:55.876783 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:55.876761    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49f521e0eb96ffa477f62f278bc4f4c4.slice/crio-e97870778f818b867ed9e13722336896b44a3893025ebc30bd0d0c45eaee109e WatchSource:0}: Error finding container e97870778f818b867ed9e13722336896b44a3893025ebc30bd0d0c45eaee109e: Status 404 returned error can't find the container with id e97870778f818b867ed9e13722336896b44a3893025ebc30bd0d0c45eaee109e
Sep 10 14:19:55.880594 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.880571    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:55.881776 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.881760    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:19:55.889984 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.889966    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:19:55.890954 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.890940    2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal"
Sep 10 14:19:55.900126 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:55.900108    2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 10 14:19:56.003808 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.003720    2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:19:56.563209 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.563175    2578 apiserver.go:52] "Watching apiserver"
Sep 10 14:19:56.568849 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.568463    2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 10 14:19:56.572860 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.571844    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-5h8xw","kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n","openshift-cluster-node-tuning-operator/tuned-228l6","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal","openshift-multus/multus-p5fzb","openshift-multus/network-metrics-daemon-xcqxv","openshift-network-diagnostics/network-check-target-5hvxb","kube-system/global-pull-secret-syncer-4lk2v","openshift-dns/node-resolver-kk5vd","openshift-image-registry/node-ca-cdx6z","openshift-multus/multus-additional-cni-plugins-q24dz","openshift-network-operator/iptables-alerter-r5756","openshift-ovn-kubernetes/ovnkube-node-7wph9"]
Sep 10 14:19:56.575284 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.575257    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:56.575427 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.575343    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:19:56.577597 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.577575    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.579645 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.579619    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-t8v7h\""
Sep 10 14:19:56.580106 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.580081    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 10 14:19:56.580274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.580257    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 10 14:19:56.580543 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.580524    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 10 14:19:56.581877 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.580997    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.581877 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.581304    2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 10 14:19:56.582664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.582441    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 10 14:19:56.582664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.582480    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 10 14:19:56.583081 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.582955    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-lh5rx\""
Sep 10 14:19:56.583162 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.583147    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 10 14:19:56.583603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.583579    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.585070 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.585045    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:19:56.585369 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.585345    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-7wlvw\""
Sep 10 14:19:56.585601 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.585584    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 10 14:19:56.586221 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.586197    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.590851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.589914    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 10 14:19:56.590851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.590030    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 10 14:19:56.590851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.590081    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-jq5qb\""
Sep 10 14:19:56.590851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.590149    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 10 14:19:56.590851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.590237    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 10 14:19:56.590851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.590272    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 10 14:19:56.590851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.590371    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 10 14:19:56.590851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.590418    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.591560 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.591542    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.591710 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.591619    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592274    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-tuned\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592347    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98x7v\" (UniqueName: \"kubernetes.io/projected/41226c18-f492-4f5f-8e28-4fa1ef96f59d-kube-api-access-98x7v\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592419    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-device-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592449    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lmdv5\" (UniqueName: \"kubernetes.io/projected/f018726e-dbf1-4a97-906d-bb814946a60f-kube-api-access-lmdv5\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysctl-conf\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592518    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-systemd\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592565    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqbx2\" (UniqueName: \"kubernetes.io/projected/ad4815b3-1e11-427b-86f1-f65145c8cf90-kube-api-access-pqbx2\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592598    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysconfig\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.592670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592678    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysctl-d\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592703    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-lib-modules\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592726    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-host\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592749    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad4815b3-1e11-427b-86f1-f65145c8cf90-serviceca\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592773    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-socket-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592836    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41226c18-f492-4f5f-8e28-4fa1ef96f59d-tmp\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-modprobe-d\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-sys-fs\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592908    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-run\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592931    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-sys\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-var-lib-kubelet\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.592979    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.593001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad4815b3-1e11-427b-86f1-f65145c8cf90-host\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.593025    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-registration-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.593062    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z68bs\" (UniqueName: \"kubernetes.io/projected/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-kube-api-access-z68bs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:56.593274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.593087    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-kubernetes\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.595367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.593382    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-2hcfz\""
Sep 10 14:19:56.595367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.594027    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 10 14:19:56.595367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.594439    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 10 14:19:56.595367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.594527    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.595367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.594723    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 10 14:19:56.595367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.594861    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 10 14:19:56.597284 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.597266    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:19:56.597376 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.597330    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:19:56.599840 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.599789    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.600361 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.600330    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 10 14:19:56.600603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.600582    2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 10 14:19:56.601106 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.601089    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wksrg\""
Sep 10 14:19:56.601229 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.601109    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:19:56.601300 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.601252    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 10 14:19:56.601663 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.601642    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-6gnkr\""
Sep 10 14:19:56.601741 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.601642    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 10 14:19:56.602185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.602157    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:19:56.602704 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.602659    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 10 14:19:56.604532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.604509    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 10 14:19:56.604634 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.604598    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-m7jbv\""
Sep 10 14:19:56.604634 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.604617    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 10 14:19:56.604997 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.604975    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.606883 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.606863    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 10 14:19:56.608210 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.607082    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 10 14:19:56.608210 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.607269    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-shvcc\""
Sep 10 14:19:56.608210 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.607375    2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 10 14:19:56.614770 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.614751    2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hz6pm"
Sep 10 14:19:56.626999 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.626140    2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hz6pm"
Sep 10 14:19:56.682356 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.682328    2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 10 14:19:56.693513 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693482    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-sys\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.693657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693530    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrkqp\" (UniqueName: \"kubernetes.io/projected/1315f507-f0bd-46a5-afa4-749348c842e8-kube-api-access-xrkqp\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.693657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-registration-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.693657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693597    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbwpk\" (UniqueName: \"kubernetes.io/projected/557f2012-6385-4f5a-94b6-ad1279da46f5-kube-api-access-bbwpk\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.693657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693621    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-kubelet\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.693657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693645    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98x7v\" (UniqueName: \"kubernetes.io/projected/41226c18-f492-4f5f-8e28-4fa1ef96f59d-kube-api-access-98x7v\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693669    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-ovn\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693692    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-hostroot\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-daemon-config\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693742    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxpmt\" (UniqueName: \"kubernetes.io/projected/102ce710-afeb-41dd-b44d-4758f577ad1b-kube-api-access-hxpmt\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/607b65d4-332c-465a-92af-3363fc98146b-tmp-dir\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693793    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lmdv5\" (UniqueName: \"kubernetes.io/projected/f018726e-dbf1-4a97-906d-bb814946a60f-kube-api-access-lmdv5\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693818    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693862    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-env-overrides\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/557f2012-6385-4f5a-94b6-ad1279da46f5-iptables-alerter-script\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693962    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.693996    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.694018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694022    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-node-log\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-sys\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694050    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694092    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/102ce710-afeb-41dd-b44d-4758f577ad1b-cni-binary-copy\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694126    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-k8s-cni-cncf-io\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694153    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/717ed4b3-2706-4ed4-a2cb-dae3040d7e30-agent-certs\") pod \"konnectivity-agent-5h8xw\" (UID: \"717ed4b3-2706-4ed4-a2cb-dae3040d7e30\") " pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/717ed4b3-2706-4ed4-a2cb-dae3040d7e30-konnectivity-ca\") pod \"konnectivity-agent-5h8xw\" (UID: \"717ed4b3-2706-4ed4-a2cb-dae3040d7e30\") " pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-lib-modules\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-kubelet\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694220    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-var-lib-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694237    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-etc-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694254    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bl44\" (UniqueName: \"kubernetes.io/projected/ea3d2523-27ca-4096-992c-beabc932cb0c-kube-api-access-7bl44\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694269    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-etc-kubernetes\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694283    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-ovnkube-config\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694299    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-modprobe-d\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694313    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-netns\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-cnibin\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.694609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694352    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-run\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694368    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-var-lib-kubelet\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-systemd-units\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694407    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/557f2012-6385-4f5a-94b6-ad1279da46f5-host-slash\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694465    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-registration-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694485    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-cnibin\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694495    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.694595    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.694665    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:19:57.194640131 +0000 UTC m=+2.962362912 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694841    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-lib-modules\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694935    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad4815b3-1e11-427b-86f1-f65145c8cf90-host\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.694987    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-modprobe-d\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-run\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695149    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-var-lib-kubelet\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695195    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-systemd\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695212    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-log-socket\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.695466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-conf-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695265    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-multus-certs\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695286    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z68bs\" (UniqueName: \"kubernetes.io/projected/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-kube-api-access-z68bs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695310    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-kubernetes\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695332    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-tuned\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695347    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-cni-netd\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695391    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-ovnkube-script-lib\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695406    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-cni-multus\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695422    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695437    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-device-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695480    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysctl-conf\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695503    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-systemd\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695526    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea3d2523-27ca-4096-992c-beabc932cb0c-ovn-node-metrics-cert\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695551    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqbx2\" (UniqueName: \"kubernetes.io/projected/ad4815b3-1e11-427b-86f1-f65145c8cf90-kube-api-access-pqbx2\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-run-netns\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.696247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695670    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad4815b3-1e11-427b-86f1-f65145c8cf90-host\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695711    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-device-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695870    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysctl-conf\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695880    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-kubernetes\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-cni-bin\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695916    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-systemd\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.695961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysconfig\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696067    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysconfig\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysctl-d\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696121    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-etc-selinux\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696107    2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696163    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-host\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696129    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-host\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-sysctl-d\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696228    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-slash\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696255    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696279    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-dbus\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.696928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696303    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad4815b3-1e11-427b-86f1-f65145c8cf90-serviceca\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-socket-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41226c18-f492-4f5f-8e28-4fa1ef96f59d-tmp\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696508    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-socket-dir\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696542    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696576    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/607b65d4-332c-465a-92af-3363fc98146b-hosts-file\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696607    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd56j\" (UniqueName: \"kubernetes.io/projected/607b65d4-332c-465a-92af-3363fc98146b-kube-api-access-wd56j\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-cni-bin\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-system-cni-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696674    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-cni-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-socket-dir-parent\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696712    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-kubelet-config\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696739    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-system-cni-dir\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-sys-fs\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696823    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ad4815b3-1e11-427b-86f1-f65145c8cf90-serviceca\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.697925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.696897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f018726e-dbf1-4a97-906d-bb814946a60f-sys-fs\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.701178 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.700880    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/41226c18-f492-4f5f-8e28-4fa1ef96f59d-etc-tuned\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.701178 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.701128    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/41226c18-f492-4f5f-8e28-4fa1ef96f59d-tmp\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.704533 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.704454    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98x7v\" (UniqueName: \"kubernetes.io/projected/41226c18-f492-4f5f-8e28-4fa1ef96f59d-kube-api-access-98x7v\") pod \"tuned-228l6\" (UID: \"41226c18-f492-4f5f-8e28-4fa1ef96f59d\") " pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.704533 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.704488    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lmdv5\" (UniqueName: \"kubernetes.io/projected/f018726e-dbf1-4a97-906d-bb814946a60f-kube-api-access-lmdv5\") pod \"aws-ebs-csi-driver-node-k547n\" (UID: \"f018726e-dbf1-4a97-906d-bb814946a60f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.704927 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.704897    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z68bs\" (UniqueName: \"kubernetes.io/projected/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-kube-api-access-z68bs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:56.706068 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.706044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqbx2\" (UniqueName: \"kubernetes.io/projected/ad4815b3-1e11-427b-86f1-f65145c8cf90-kube-api-access-pqbx2\") pod \"node-ca-cdx6z\" (UID: \"ad4815b3-1e11-427b-86f1-f65145c8cf90\") " pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.724085 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.724029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal" event={"ID":"49f521e0eb96ffa477f62f278bc4f4c4","Type":"ContainerStarted","Data":"e97870778f818b867ed9e13722336896b44a3893025ebc30bd0d0c45eaee109e"}
Sep 10 14:19:56.725585 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.725558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal" event={"ID":"579903845495bf197e8392d11f220146","Type":"ContainerStarted","Data":"562673dd0871b1ea1a75b73a16ba941e8694241acd596daee42f8f82b2a4ba6e"}
Sep 10 14:19:56.797267 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/102ce710-afeb-41dd-b44d-4758f577ad1b-cni-binary-copy\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.797267 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797280    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-k8s-cni-cncf-io\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797321    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/717ed4b3-2706-4ed4-a2cb-dae3040d7e30-agent-certs\") pod \"konnectivity-agent-5h8xw\" (UID: \"717ed4b3-2706-4ed4-a2cb-dae3040d7e30\") " pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797346    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/717ed4b3-2706-4ed4-a2cb-dae3040d7e30-konnectivity-ca\") pod \"konnectivity-agent-5h8xw\" (UID: \"717ed4b3-2706-4ed4-a2cb-dae3040d7e30\") " pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797372    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-kubelet\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797395    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-var-lib-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797419    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-etc-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bl44\" (UniqueName: \"kubernetes.io/projected/ea3d2523-27ca-4096-992c-beabc932cb0c-kube-api-access-7bl44\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797468    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-etc-kubernetes\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797492    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-ovnkube-config\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797519    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-netns\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.797554 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797543    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-cnibin\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797569    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-systemd-units\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/557f2012-6385-4f5a-94b6-ad1279da46f5-host-slash\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797622    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-cnibin\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-systemd\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797684    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-log-socket\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797708    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797732    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-conf-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797757    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-multus-certs\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797784    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-cni-netd\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-ovnkube-script-lib\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797853    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-cni-multus\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797877    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797903    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797933    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea3d2523-27ca-4096-992c-beabc932cb0c-ovn-node-metrics-cert\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-run-netns\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.797986    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-cni-bin\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798016    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-slash\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798064    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-dbus\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798092    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798115    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/607b65d4-332c-465a-92af-3363fc98146b-hosts-file\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798141    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd56j\" (UniqueName: \"kubernetes.io/projected/607b65d4-332c-465a-92af-3363fc98146b-kube-api-access-wd56j\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798167    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-cni-bin\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798187    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-system-cni-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-cni-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798236    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-socket-dir-parent\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798268    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-kubelet-config\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798296    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-system-cni-dir\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrkqp\" (UniqueName: \"kubernetes.io/projected/1315f507-f0bd-46a5-afa4-749348c842e8-kube-api-access-xrkqp\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bbwpk\" (UniqueName: \"kubernetes.io/projected/557f2012-6385-4f5a-94b6-ad1279da46f5-kube-api-access-bbwpk\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-kubelet\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798413    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-ovn\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798440    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-hostroot\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.798745 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798466    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-daemon-config\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxpmt\" (UniqueName: \"kubernetes.io/projected/102ce710-afeb-41dd-b44d-4758f577ad1b-kube-api-access-hxpmt\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798515    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/607b65d4-332c-465a-92af-3363fc98146b-tmp-dir\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798541    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798568    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-env-overrides\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798593    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/557f2012-6385-4f5a-94b6-ad1279da46f5-iptables-alerter-script\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798647    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-node-log\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798707    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.799359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.798813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.799757 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.799384    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/102ce710-afeb-41dd-b44d-4758f577ad1b-cni-binary-copy\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.799757 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.799433    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-k8s-cni-cncf-io\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.799871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.799837    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-slash\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-systemd\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800057    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-etc-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800126    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-kubelet\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.800157    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800170    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-var-lib-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800064    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-system-cni-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800180    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-netns\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800220    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-cnibin\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.800229    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret podName:3779e8a4-b29e-4703-846e-1a6a3b58c4cd nodeName:}" failed. No retries permitted until 2026-09-10 14:19:57.300209901 +0000 UTC m=+3.067932662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret") pod "global-pull-secret-syncer-4lk2v" (UID: "3779e8a4-b29e-4703-846e-1a6a3b58c4cd") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800274    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-systemd-units\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800308    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/557f2012-6385-4f5a-94b6-ad1279da46f5-host-slash\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-cnibin\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800360    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/717ed4b3-2706-4ed4-a2cb-dae3040d7e30-konnectivity-ca\") pod \"konnectivity-agent-5h8xw\" (UID: \"717ed4b3-2706-4ed4-a2cb-dae3040d7e30\") " pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800379    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-cni-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800414    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/607b65d4-332c-465a-92af-3363fc98146b-hosts-file\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800419    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-cni-bin\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800443    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-system-cni-dir\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.800974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-dbus\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-kubelet-config\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800527    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-ovnkube-config\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-socket-dir-parent\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800578    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/1315f507-f0bd-46a5-afa4-749348c842e8-tuning-conf-dir\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-cni-bin\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-cni-multus\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800722    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-etc-kubernetes\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800763    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-hostroot\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-ovnkube-script-lib\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800794    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-run-netns\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800805    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-var-lib-kubelet\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800851    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-log-socket\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800862    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-ovn\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800878    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/607b65d4-332c-465a-92af-3363fc98146b-tmp-dir\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-conf-dir\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/102ce710-afeb-41dd-b44d-4758f577ad1b-host-run-multus-certs\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.800955    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-cni-netd\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.801966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.801151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-run-openvswitch\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.802762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.801203    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-node-log\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.802762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.801281    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/557f2012-6385-4f5a-94b6-ad1279da46f5-iptables-alerter-script\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.802762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.801363    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/102ce710-afeb-41dd-b44d-4758f577ad1b-multus-daemon-config\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.802762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.801459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ea3d2523-27ca-4096-992c-beabc932cb0c-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.802762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.802008    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.802762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.802257    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.802762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.802332    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/1315f507-f0bd-46a5-afa4-749348c842e8-cni-binary-copy\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.802762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.802423    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ea3d2523-27ca-4096-992c-beabc932cb0c-env-overrides\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.803524 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.803494    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ea3d2523-27ca-4096-992c-beabc932cb0c-ovn-node-metrics-cert\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.804387 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.804365    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/717ed4b3-2706-4ed4-a2cb-dae3040d7e30-agent-certs\") pod \"konnectivity-agent-5h8xw\" (UID: \"717ed4b3-2706-4ed4-a2cb-dae3040d7e30\") " pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:19:56.807508 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.807090    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:19:56.807508 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.807116    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:19:56.807508 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.807130    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8nxcm for pod openshift-network-diagnostics/network-check-target-5hvxb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:19:56.807508 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:56.807204    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm podName:d1a9479e-dc19-4171-ac1b-f6e95fe6841c nodeName:}" failed. No retries permitted until 2026-09-10 14:19:57.307186197 +0000 UTC m=+3.074908968 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-8nxcm" (UniqueName: "kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm") pod "network-check-target-5hvxb" (UID: "d1a9479e-dc19-4171-ac1b-f6e95fe6841c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:19:56.808884 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.808860    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbwpk\" (UniqueName: \"kubernetes.io/projected/557f2012-6385-4f5a-94b6-ad1279da46f5-kube-api-access-bbwpk\") pod \"iptables-alerter-r5756\" (UID: \"557f2012-6385-4f5a-94b6-ad1279da46f5\") " pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.809287 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.809253    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd56j\" (UniqueName: \"kubernetes.io/projected/607b65d4-332c-465a-92af-3363fc98146b-kube-api-access-wd56j\") pod \"node-resolver-kk5vd\" (UID: \"607b65d4-332c-465a-92af-3363fc98146b\") " pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.810818 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.810789    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bl44\" (UniqueName: \"kubernetes.io/projected/ea3d2523-27ca-4096-992c-beabc932cb0c-kube-api-access-7bl44\") pod \"ovnkube-node-7wph9\" (UID: \"ea3d2523-27ca-4096-992c-beabc932cb0c\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.811186 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.811168    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxpmt\" (UniqueName: \"kubernetes.io/projected/102ce710-afeb-41dd-b44d-4758f577ad1b-kube-api-access-hxpmt\") pod \"multus-p5fzb\" (UID: \"102ce710-afeb-41dd-b44d-4758f577ad1b\") " pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.812305 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.812263    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrkqp\" (UniqueName: \"kubernetes.io/projected/1315f507-f0bd-46a5-afa4-749348c842e8-kube-api-access-xrkqp\") pod \"multus-additional-cni-plugins-q24dz\" (UID: \"1315f507-f0bd-46a5-afa4-749348c842e8\") " pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:56.893377 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.893281    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-cdx6z"
Sep 10 14:19:56.903926 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.903894    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n"
Sep 10 14:19:56.919117 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.919085    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-228l6"
Sep 10 14:19:56.923216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.923190    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:19:56.930230 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.930209    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-p5fzb"
Sep 10 14:19:56.940360 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.940332    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-r5756"
Sep 10 14:19:56.949262 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.949234    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kk5vd"
Sep 10 14:19:56.959267 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.959247    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:19:56.965986 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:56.965967    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-q24dz"
Sep 10 14:19:57.203433 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:57.203336    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:57.203625 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.203508    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:19:57.203625 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.203600    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:19:58.203577783 +0000 UTC m=+3.971300547 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:19:57.303784 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:57.303743    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:57.303956 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.303921    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:19:57.304009 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.304000    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret podName:3779e8a4-b29e-4703-846e-1a6a3b58c4cd nodeName:}" failed. No retries permitted until 2026-09-10 14:19:58.303981882 +0000 UTC m=+4.071704652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret") pod "global-pull-secret-syncer-4lk2v" (UID: "3779e8a4-b29e-4703-846e-1a6a3b58c4cd") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:19:57.405118 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:57.405072    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:19:57.405294 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.405262    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:19:57.405294 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.405283    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:19:57.405294 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.405294    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8nxcm for pod openshift-network-diagnostics/network-check-target-5hvxb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:19:57.405408 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.405358    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm podName:d1a9479e-dc19-4171-ac1b-f6e95fe6841c nodeName:}" failed. No retries permitted until 2026-09-10 14:19:58.405340456 +0000 UTC m=+4.173063241 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-8nxcm" (UniqueName: "kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm") pod "network-check-target-5hvxb" (UID: "d1a9479e-dc19-4171-ac1b-f6e95fe6841c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:19:57.629239 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:57.629155    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:14:57 +0000 UTC" deadline="2028-03-27 06:00:46.661364343 +0000 UTC"
Sep 10 14:19:57.629239 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:57.629202    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13527h40m49.032180795s"
Sep 10 14:19:57.718112 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:57.718078    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:57.718283 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:57.718224    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:19:57.996290 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:57.996254    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod557f2012_6385_4f5a_94b6_ad1279da46f5.slice/crio-473a34ab6a85734b043d91e4787967e843279da911689cb53db169ac65c2b5b4 WatchSource:0}: Error finding container 473a34ab6a85734b043d91e4787967e843279da911689cb53db169ac65c2b5b4: Status 404 returned error can't find the container with id 473a34ab6a85734b043d91e4787967e843279da911689cb53db169ac65c2b5b4
Sep 10 14:19:57.996678 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:57.996655    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod102ce710_afeb_41dd_b44d_4758f577ad1b.slice/crio-9d3086318ebb2c85fa2d3a9e9ea59716041967e93fbaa2235e1b260f4e062734 WatchSource:0}: Error finding container 9d3086318ebb2c85fa2d3a9e9ea59716041967e93fbaa2235e1b260f4e062734: Status 404 returned error can't find the container with id 9d3086318ebb2c85fa2d3a9e9ea59716041967e93fbaa2235e1b260f4e062734
Sep 10 14:19:57.998343 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:57.998002    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41226c18_f492_4f5f_8e28_4fa1ef96f59d.slice/crio-d7d63bf256ecbe7937a5c5712edd9654adcd1db6e9d2ff01d5485965077eec3f WatchSource:0}: Error finding container d7d63bf256ecbe7937a5c5712edd9654adcd1db6e9d2ff01d5485965077eec3f: Status 404 returned error can't find the container with id d7d63bf256ecbe7937a5c5712edd9654adcd1db6e9d2ff01d5485965077eec3f
Sep 10 14:19:57.998409 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:57.998337    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1315f507_f0bd_46a5_afa4_749348c842e8.slice/crio-7ab161934883d88106354f5ae0137305f3f75371c6a611008d66b0cf2271e175 WatchSource:0}: Error finding container 7ab161934883d88106354f5ae0137305f3f75371c6a611008d66b0cf2271e175: Status 404 returned error can't find the container with id 7ab161934883d88106354f5ae0137305f3f75371c6a611008d66b0cf2271e175
Sep 10 14:19:58.003249 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:58.003236    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf018726e_dbf1_4a97_906d_bb814946a60f.slice/crio-6579e1555e7e48e6b98b7e3d9486ec81244236bc591ba1e06466fb3dab9b180b WatchSource:0}: Error finding container 6579e1555e7e48e6b98b7e3d9486ec81244236bc591ba1e06466fb3dab9b180b: Status 404 returned error can't find the container with id 6579e1555e7e48e6b98b7e3d9486ec81244236bc591ba1e06466fb3dab9b180b
Sep 10 14:19:58.003440 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:58.003415    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea3d2523_27ca_4096_992c_beabc932cb0c.slice/crio-3b099cf003814f6677b10d0b4b1f1ae50a42efb3b404c0c9f26a0e06efa99718 WatchSource:0}: Error finding container 3b099cf003814f6677b10d0b4b1f1ae50a42efb3b404c0c9f26a0e06efa99718: Status 404 returned error can't find the container with id 3b099cf003814f6677b10d0b4b1f1ae50a42efb3b404c0c9f26a0e06efa99718
Sep 10 14:19:58.004099 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:58.003979    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod717ed4b3_2706_4ed4_a2cb_dae3040d7e30.slice/crio-0f27394ca01ea0684c09a7b66ed290fcaa61e6a3b9fe14078c8c388875686363 WatchSource:0}: Error finding container 0f27394ca01ea0684c09a7b66ed290fcaa61e6a3b9fe14078c8c388875686363: Status 404 returned error can't find the container with id 0f27394ca01ea0684c09a7b66ed290fcaa61e6a3b9fe14078c8c388875686363
Sep 10 14:19:58.004780 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:58.004706    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad4815b3_1e11_427b_86f1_f65145c8cf90.slice/crio-c1c482bd9dc3aebdb6fe24e6ae6a1a9e102f2512b7316288125dc5f3e29daa17 WatchSource:0}: Error finding container c1c482bd9dc3aebdb6fe24e6ae6a1a9e102f2512b7316288125dc5f3e29daa17: Status 404 returned error can't find the container with id c1c482bd9dc3aebdb6fe24e6ae6a1a9e102f2512b7316288125dc5f3e29daa17
Sep 10 14:19:58.008747 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:19:58.006929    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod607b65d4_332c_465a_92af_3363fc98146b.slice/crio-70a6baeed3199dc8ad53468e011d08e7a8b84419251407f988f670b7d5309df4 WatchSource:0}: Error finding container 70a6baeed3199dc8ad53468e011d08e7a8b84419251407f988f670b7d5309df4: Status 404 returned error can't find the container with id 70a6baeed3199dc8ad53468e011d08e7a8b84419251407f988f670b7d5309df4
Sep 10 14:19:58.211017 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.210980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:58.211169 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.211145    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:19:58.211248 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.211228    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:20:00.211203817 +0000 UTC m=+5.978926599 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:19:58.311646 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.311619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:58.311784 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.311770    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:19:58.311842 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.311830    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret podName:3779e8a4-b29e-4703-846e-1a6a3b58c4cd nodeName:}" failed. No retries permitted until 2026-09-10 14:20:00.311812022 +0000 UTC m=+6.079534782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret") pod "global-pull-secret-syncer-4lk2v" (UID: "3779e8a4-b29e-4703-846e-1a6a3b58c4cd") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:19:58.412685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.412624    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:19:58.412891 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.412866    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:19:58.412950 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.412899    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:19:58.412950 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.412915    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8nxcm for pod openshift-network-diagnostics/network-check-target-5hvxb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:19:58.413048 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.412983    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm podName:d1a9479e-dc19-4171-ac1b-f6e95fe6841c nodeName:}" failed. No retries permitted until 2026-09-10 14:20:00.412962154 +0000 UTC m=+6.180684917 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-8nxcm" (UniqueName: "kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm") pod "network-check-target-5hvxb" (UID: "d1a9479e-dc19-4171-ac1b-f6e95fe6841c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:19:58.630311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.630152    2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-09 14:14:57 +0000 UTC" deadline="2028-02-19 13:03:20.522502473 +0000 UTC"
Sep 10 14:19:58.630311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.630227    2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12646h43m21.892280449s"
Sep 10 14:19:58.720580 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.720519    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:19:58.720768 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.720659    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:19:58.721143 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.721096    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:19:58.721266 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:58.721200    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:19:58.738766 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.738718    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n" event={"ID":"f018726e-dbf1-4a97-906d-bb814946a60f","Type":"ContainerStarted","Data":"6579e1555e7e48e6b98b7e3d9486ec81244236bc591ba1e06466fb3dab9b180b"}
Sep 10 14:19:58.746006 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.745161    2578 generic.go:356] "Generic (PLEG): container finished" podID="49f521e0eb96ffa477f62f278bc4f4c4" containerID="f98002fff84b43a9ca022d42e3d40add56f0772f5f4878c07e8369e933b5dc8f" exitCode=0
Sep 10 14:19:58.746006 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.745378    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal" event={"ID":"49f521e0eb96ffa477f62f278bc4f4c4","Type":"ContainerDied","Data":"f98002fff84b43a9ca022d42e3d40add56f0772f5f4878c07e8369e933b5dc8f"}
Sep 10 14:19:58.750033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.749983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kk5vd" event={"ID":"607b65d4-332c-465a-92af-3363fc98146b","Type":"ContainerStarted","Data":"70a6baeed3199dc8ad53468e011d08e7a8b84419251407f988f670b7d5309df4"}
Sep 10 14:19:58.758573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.758545    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cdx6z" event={"ID":"ad4815b3-1e11-427b-86f1-f65145c8cf90","Type":"ContainerStarted","Data":"c1c482bd9dc3aebdb6fe24e6ae6a1a9e102f2512b7316288125dc5f3e29daa17"}
Sep 10 14:19:58.772221 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.772057    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q24dz" event={"ID":"1315f507-f0bd-46a5-afa4-749348c842e8","Type":"ContainerStarted","Data":"7ab161934883d88106354f5ae0137305f3f75371c6a611008d66b0cf2271e175"}
Sep 10 14:19:58.792410 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.792366    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-228l6" event={"ID":"41226c18-f492-4f5f-8e28-4fa1ef96f59d","Type":"ContainerStarted","Data":"d7d63bf256ecbe7937a5c5712edd9654adcd1db6e9d2ff01d5485965077eec3f"}
Sep 10 14:19:58.803123 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.801073    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p5fzb" event={"ID":"102ce710-afeb-41dd-b44d-4758f577ad1b","Type":"ContainerStarted","Data":"9d3086318ebb2c85fa2d3a9e9ea59716041967e93fbaa2235e1b260f4e062734"}
Sep 10 14:19:58.806088 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.806053    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r5756" event={"ID":"557f2012-6385-4f5a-94b6-ad1279da46f5","Type":"ContainerStarted","Data":"473a34ab6a85734b043d91e4787967e843279da911689cb53db169ac65c2b5b4"}
Sep 10 14:19:58.809481 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.809372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal" event={"ID":"579903845495bf197e8392d11f220146","Type":"ContainerStarted","Data":"3fa767d54e654fa15d4c03010f6c846cd93dbeb628b0e6f00a680576ff454e6d"}
Sep 10 14:19:58.820291 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.820230    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5h8xw" event={"ID":"717ed4b3-2706-4ed4-a2cb-dae3040d7e30","Type":"ContainerStarted","Data":"0f27394ca01ea0684c09a7b66ed290fcaa61e6a3b9fe14078c8c388875686363"}
Sep 10 14:19:58.825227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.824886    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-68.ec2.internal" podStartSLOduration=2.8248652659999998 podStartE2EDuration="2.824865266s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:58.823584864 +0000 UTC m=+4.591307647" watchObservedRunningTime="2026-09-10 14:19:58.824865266 +0000 UTC m=+4.592588050"
Sep 10 14:19:58.842239 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:58.842198    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"3b099cf003814f6677b10d0b4b1f1ae50a42efb3b404c0c9f26a0e06efa99718"}
Sep 10 14:19:59.718237 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:59.718038    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:19:59.718237 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:19:59.718188    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:19:59.872930 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:59.872892    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal" event={"ID":"49f521e0eb96ffa477f62f278bc4f4c4","Type":"ContainerStarted","Data":"d6aa71b63cedbea974f0c50451865725a9d7c1241e0a76fc7198265b838bc632"}
Sep 10 14:19:59.887796 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:19:59.887734    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-68.ec2.internal" podStartSLOduration=3.887711814 podStartE2EDuration="3.887711814s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:19:59.887338665 +0000 UTC m=+5.655061452" watchObservedRunningTime="2026-09-10 14:19:59.887711814 +0000 UTC m=+5.655434598"
Sep 10 14:20:00.233185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:00.233119    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:00.233387 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.233303    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:20:00.233387 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.233385    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:20:04.233362816 +0000 UTC m=+10.001085595 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:20:00.334020 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:00.333985    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:00.334229 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.334155    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:20:00.334229 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.334223    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret podName:3779e8a4-b29e-4703-846e-1a6a3b58c4cd nodeName:}" failed. No retries permitted until 2026-09-10 14:20:04.334203786 +0000 UTC m=+10.101926550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret") pod "global-pull-secret-syncer-4lk2v" (UID: "3779e8a4-b29e-4703-846e-1a6a3b58c4cd") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:20:00.435245 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:00.435203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:00.435485 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.435404    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:20:00.435485 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.435431    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:20:00.435485 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.435446    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8nxcm for pod openshift-network-diagnostics/network-check-target-5hvxb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:20:00.435604 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.435512    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm podName:d1a9479e-dc19-4171-ac1b-f6e95fe6841c nodeName:}" failed. No retries permitted until 2026-09-10 14:20:04.435491511 +0000 UTC m=+10.203214291 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-8nxcm" (UniqueName: "kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm") pod "network-check-target-5hvxb" (UID: "d1a9479e-dc19-4171-ac1b-f6e95fe6841c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:20:00.720916 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:00.720809    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:00.721367 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.720976    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:00.721367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:00.721029    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:00.721367 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:00.721104    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:01.718099 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:01.718051    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:01.718317 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:01.718222    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:02.729398 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:02.729340    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:02.729922 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:02.729475    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:02.729922 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:02.729919    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:02.730067 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:02.729985    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:03.718716 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:03.718621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:03.718915 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:03.718807    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:04.269252 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:04.269207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:04.269729 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.269461    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:20:04.269729 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.269533    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:20:12.269512633 +0000 UTC m=+18.037235410 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:20:04.370360 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:04.370316    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:04.370561 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.370515    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:20:04.370628 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.370572    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret podName:3779e8a4-b29e-4703-846e-1a6a3b58c4cd nodeName:}" failed. No retries permitted until 2026-09-10 14:20:12.370557387 +0000 UTC m=+18.138280147 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret") pod "global-pull-secret-syncer-4lk2v" (UID: "3779e8a4-b29e-4703-846e-1a6a3b58c4cd") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:20:04.471432 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:04.471389    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:04.471639 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.471595    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:20:04.471639 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.471630    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:20:04.471639 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.471641    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8nxcm for pod openshift-network-diagnostics/network-check-target-5hvxb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:20:04.471877 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.471725    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm podName:d1a9479e-dc19-4171-ac1b-f6e95fe6841c nodeName:}" failed. No retries permitted until 2026-09-10 14:20:12.471702864 +0000 UTC m=+18.239425630 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-8nxcm" (UniqueName: "kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm") pod "network-check-target-5hvxb" (UID: "d1a9479e-dc19-4171-ac1b-f6e95fe6841c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:20:04.719750 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:04.719673    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:04.719933 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.719813    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:04.720364 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:04.720343    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:04.720472 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:04.720451    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:05.718246 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:05.718212    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:05.718700 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:05.718342    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:06.718548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:06.718516    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:06.719040 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:06.718527    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:06.719040 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:06.718670    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:06.719040 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:06.718730    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:07.718713 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:07.718671    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:07.719209 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:07.718828    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:08.718370 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:08.718333    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:08.718552 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:08.718441    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:08.718552 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:08.718503    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:08.718676 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:08.718615    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:09.718060 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:09.718022    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:09.718517 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:09.718165    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:10.718655 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:10.718621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:10.719122 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:10.718763    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:10.719122 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:10.718776    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:10.719122 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:10.718866    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:11.717967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:11.717924    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:11.718206 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:11.718074    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:12.335243 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:12.335208    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:12.335684 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.335383    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:20:12.335684 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.335454    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:20:28.335436406 +0000 UTC m=+34.103159164 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:20:12.436517 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:12.436477    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:12.436695 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.436617    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:20:12.436695 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.436694    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret podName:3779e8a4-b29e-4703-846e-1a6a3b58c4cd nodeName:}" failed. No retries permitted until 2026-09-10 14:20:28.436671979 +0000 UTC m=+34.204394760 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret") pod "global-pull-secret-syncer-4lk2v" (UID: "3779e8a4-b29e-4703-846e-1a6a3b58c4cd") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:20:12.537822 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:12.537785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:12.538007 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.537964    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:20:12.538007 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.537986    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:20:12.538007 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.537996    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8nxcm for pod openshift-network-diagnostics/network-check-target-5hvxb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:20:12.538156 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.538048    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm podName:d1a9479e-dc19-4171-ac1b-f6e95fe6841c nodeName:}" failed. No retries permitted until 2026-09-10 14:20:28.538035409 +0000 UTC m=+34.305758167 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-8nxcm" (UniqueName: "kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm") pod "network-check-target-5hvxb" (UID: "d1a9479e-dc19-4171-ac1b-f6e95fe6841c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:20:12.718253 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:12.718153    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:12.718253 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:12.718173    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:12.718434 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.718280    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:12.718434 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:12.718394    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:13.718468 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:13.718430    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:13.718955 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:13.718556    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:14.719414 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:14.719378    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:14.719849 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:14.719499    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:14.719849 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:14.719538    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:14.719849 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:14.719610    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:15.717998 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:15.717947    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:15.718191 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:15.718100    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:16.719068 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.718571    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:16.720016 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.718630    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:16.720016 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:16.719171    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:16.720016 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:16.719255    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:16.919678 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.919637    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-5h8xw" event={"ID":"717ed4b3-2706-4ed4-a2cb-dae3040d7e30","Type":"ContainerStarted","Data":"b1fa03f55008937f67b295daf8be6ff7740d8bf0d9b94e6c99c96f1e5e3ab9d0"}
Sep 10 14:20:16.923432 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.923388    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"4e1b0c34ebc8fb91fa40e5c3ceef4994981e52a337bf321d544cb32fce5c3d1b"}
Sep 10 14:20:16.923581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.923467    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"160e9be7315d64989b47e8af8a7110b5477495343dfde2e2a8ee9952bf2650f5"}
Sep 10 14:20:16.923581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.923510    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"62ffb31a049e5db80d60df5d3234e15de0b0b927f596cf9a99dde50479e73713"}
Sep 10 14:20:16.923581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.923555    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"2029723b1dbd52d08fa770299a253c55802969a6a06b7fc8417e3c42b1d0cc75"}
Sep 10 14:20:16.923581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.923583    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"d3f731a27a917f6bf210b9c6b69acaa74ad7b3698f675a5218ec50209db76cdf"}
Sep 10 14:20:16.923799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.923624    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"a22b876dd2248bab0a9e27fba0981396abed14a072c9e11926b83feb64405940"}
Sep 10 14:20:16.924980 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.924957    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n" event={"ID":"f018726e-dbf1-4a97-906d-bb814946a60f","Type":"ContainerStarted","Data":"01e9d787ad947f145699445bee4772579b2bc936bcd3589dadf9de2f727caab7"}
Sep 10 14:20:16.926380 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.926348    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kk5vd" event={"ID":"607b65d4-332c-465a-92af-3363fc98146b","Type":"ContainerStarted","Data":"7caadd2c96f09a3ea1570f6d3da009a81c0d670f0492cb59a88e468ae5641400"}
Sep 10 14:20:16.927971 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.927819    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-cdx6z" event={"ID":"ad4815b3-1e11-427b-86f1-f65145c8cf90","Type":"ContainerStarted","Data":"2da95f7ccd0fad49270369696bc268b90720249b8f8795665c85de5673e1bfc2"}
Sep 10 14:20:16.929390 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.929362    2578 generic.go:356] "Generic (PLEG): container finished" podID="1315f507-f0bd-46a5-afa4-749348c842e8" containerID="1e5b44516c5db3254e94bc26a7f705edce5eab63801798322b7514935b8a1f6b" exitCode=0
Sep 10 14:20:16.929501 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.929394    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q24dz" event={"ID":"1315f507-f0bd-46a5-afa4-749348c842e8","Type":"ContainerDied","Data":"1e5b44516c5db3254e94bc26a7f705edce5eab63801798322b7514935b8a1f6b"}
Sep 10 14:20:16.931124 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.931094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-228l6" event={"ID":"41226c18-f492-4f5f-8e28-4fa1ef96f59d","Type":"ContainerStarted","Data":"428c33dae50fc13fe5675f20f1fd82cc86e74b70bc0165047093fd9438a4fd0d"}
Sep 10 14:20:16.932543 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.932522    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-p5fzb" event={"ID":"102ce710-afeb-41dd-b44d-4758f577ad1b","Type":"ContainerStarted","Data":"b943a5774e97a6e40ead0242b7ff9d3cc7459a64b2c99874626906ed7949dea5"}
Sep 10 14:20:16.947877 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.947781    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-cdx6z" podStartSLOduration=4.059885183 podStartE2EDuration="21.947760571s" totalImagesPullingTime="17.887875388s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:16.947358754 +0000 UTC m=+22.715081530" watchObservedRunningTime="2026-09-10 14:20:16.947760571 +0000 UTC m=+22.715483343"
Sep 10 14:20:16.948197 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.948165    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-5h8xw" podStartSLOduration=4.059209297 podStartE2EDuration="21.948155405s" totalImagesPullingTime="17.888946108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:16.934815651 +0000 UTC m=+22.702538431" watchObservedRunningTime="2026-09-10 14:20:16.948155405 +0000 UTC m=+22.715878184"
Sep 10 14:20:16.966184 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:16.966136    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-p5fzb" podStartSLOduration=3.99206105 podStartE2EDuration="21.966119443s" totalImagesPullingTime="17.974058393s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:16.965813824 +0000 UTC m=+22.733536605" watchObservedRunningTime="2026-09-10 14:20:16.966119443 +0000 UTC m=+22.733842226"
Sep 10 14:20:17.010058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.010000    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-228l6" podStartSLOduration=4.114060357 podStartE2EDuration="22.009980325s" totalImagesPullingTime="17.895919968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:17.009614847 +0000 UTC m=+22.777337629" watchObservedRunningTime="2026-09-10 14:20:17.009980325 +0000 UTC m=+22.777703104"
Sep 10 14:20:17.029535 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.029484    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kk5vd" podStartSLOduration=4.143120514 podStartE2EDuration="22.02946688s" totalImagesPullingTime="17.886346366s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:17.029100993 +0000 UTC m=+22.796823774" watchObservedRunningTime="2026-09-10 14:20:17.02946688 +0000 UTC m=+22.797189660"
Sep 10 14:20:17.587594 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.587557    2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 10 14:20:17.651298 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.651165    2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-10T14:20:17.587591438Z","UUID":"bd4b0d94-7604-47ac-b0cd-f8c60db96976","Handler":null,"Name":"","Endpoint":""}
Sep 10 14:20:17.652868 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.652843    2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 10 14:20:17.652868 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.652883    2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 10 14:20:17.717759 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.717712    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:17.717930 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:17.717860    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:17.937718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.937631    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-r5756" event={"ID":"557f2012-6385-4f5a-94b6-ad1279da46f5","Type":"ContainerStarted","Data":"409ea11ca605f822ff42fb0d90bcee9704fd18819f8f081862362d5bba888b32"}
Sep 10 14:20:17.940303 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.939929    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n" event={"ID":"f018726e-dbf1-4a97-906d-bb814946a60f","Type":"ContainerStarted","Data":"088998b0e1742039e9291baca826d9ac89fb74badaefd824b30fe34c4a90e066"}
Sep 10 14:20:17.950735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:17.950687    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-r5756" podStartSLOduration=5.044173897 podStartE2EDuration="22.950671227s" totalImagesPullingTime="17.90649733s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:17.950276969 +0000 UTC m=+23.717999748" watchObservedRunningTime="2026-09-10 14:20:17.950671227 +0000 UTC m=+23.718394006"
Sep 10 14:20:18.718089 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:18.718056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:18.718275 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:18.718174    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:18.718563 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:18.718544    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:18.718734 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:18.718713    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:18.888260 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:18.888226    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:20:18.888699 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:18.888675    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:20:18.947047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:18.946999    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"5acad728bdc76b9a2b7b6ee13d5abf46f7da9b8fc6ea3d61b4d62e3255c65d0e"}
Sep 10 14:20:18.947606 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:18.947578    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:20:18.947818 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:18.947802    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-5h8xw"
Sep 10 14:20:19.718583 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:19.718542    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:19.718801 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:19.718692    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:19.952411 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:19.952370    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n" event={"ID":"f018726e-dbf1-4a97-906d-bb814946a60f","Type":"ContainerStarted","Data":"16b2e00ccceba7751fb5196573e9fae3e1c6ace8679dfb82b6575d0df7091ede"}
Sep 10 14:20:19.969636 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:19.969538    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-k547n" podStartSLOduration=4.298825521 podStartE2EDuration="24.969521297s" totalImagesPullingTime="20.670695776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:19.968945464 +0000 UTC m=+25.736668245" watchObservedRunningTime="2026-09-10 14:20:19.969521297 +0000 UTC m=+25.737244138"
Sep 10 14:20:20.718299 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:20.718048    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:20.718473 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:20.718406    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:20.718779 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:20.718056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:20.718889 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:20.718869    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:21.718640 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:21.718555    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:21.719507 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:21.718687    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:21.961294 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:21.961263    2578 generic.go:356] "Generic (PLEG): container finished" podID="1315f507-f0bd-46a5-afa4-749348c842e8" containerID="d4367521d8c88d4505d4576e17ee2750f2c137519adddcbc8ecf92236403d074" exitCode=0
Sep 10 14:20:21.961466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:21.961362    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q24dz" event={"ID":"1315f507-f0bd-46a5-afa4-749348c842e8","Type":"ContainerDied","Data":"d4367521d8c88d4505d4576e17ee2750f2c137519adddcbc8ecf92236403d074"}
Sep 10 14:20:21.965372 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:21.965350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" event={"ID":"ea3d2523-27ca-4096-992c-beabc932cb0c","Type":"ContainerStarted","Data":"5d78534bb77c6ebc17a1d59399920a3e6323e917eef04f9b73a1149a1f325a9b"}
Sep 10 14:20:21.965633 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:21.965616    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:20:21.965683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:21.965678    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:20:21.981038 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:21.980958    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:20:21.981159 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:21.981123    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:20:22.002430 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:22.001847    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9" podStartSLOduration=9.060686253 podStartE2EDuration="27.001829892s" totalImagesPullingTime="17.941143639s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:22.001306332 +0000 UTC m=+27.769029110" watchObservedRunningTime="2026-09-10 14:20:22.001829892 +0000 UTC m=+27.769552674"
Sep 10 14:20:22.718400 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:22.718180    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:22.718603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:22.718259    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:22.718603 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:22.718510    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:22.718603 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:22.718554    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:22.969965 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:22.969879    2578 generic.go:356] "Generic (PLEG): container finished" podID="1315f507-f0bd-46a5-afa4-749348c842e8" containerID="91da29967e957363ed6157c191a0d8e80b02e5c80effe79cfaaf6f6f7e409c8c" exitCode=0
Sep 10 14:20:22.970087 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:22.969969    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q24dz" event={"ID":"1315f507-f0bd-46a5-afa4-749348c842e8","Type":"ContainerDied","Data":"91da29967e957363ed6157c191a0d8e80b02e5c80effe79cfaaf6f6f7e409c8c"}
Sep 10 14:20:22.970087 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:22.970036    2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 14:20:23.037138 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:23.037101    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xcqxv"]
Sep 10 14:20:23.037293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:23.037241    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:23.037366 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:23.037344    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:23.037847 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:23.037807    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lk2v"]
Sep 10 14:20:23.037973 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:23.037905    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:23.038031 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:23.038005    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:23.049416 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:23.049383    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5hvxb"]
Sep 10 14:20:23.049532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:23.049466    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:23.049580 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:23.049537    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:23.972041 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:23.972013    2578 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 10 14:20:24.336145 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:24.336096    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:20:24.718762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:24.718651    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:24.718905 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:24.718761    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:24.718905 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:24.718772    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:24.718905 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:24.718824    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:24.719018 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:24.718915    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:24.719018 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:24.718981    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:24.977217 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:24.977135    2578 generic.go:356] "Generic (PLEG): container finished" podID="1315f507-f0bd-46a5-afa4-749348c842e8" containerID="d4c345ca9a1f0bf95eca56f1d28eaa616c5d8b1fe19e743e0c416dbddd5b399e" exitCode=0
Sep 10 14:20:24.977521 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:24.977211    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q24dz" event={"ID":"1315f507-f0bd-46a5-afa4-749348c842e8","Type":"ContainerDied","Data":"d4c345ca9a1f0bf95eca56f1d28eaa616c5d8b1fe19e743e0c416dbddd5b399e"}
Sep 10 14:20:25.987374 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:25.987103    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:26.086103 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:26.086066    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:26.152586 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:26.152551    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:26.718054 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:26.718020    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:26.718231 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:26.718022    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:26.718231 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:26.718150    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:26.718231 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:26.718180    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:26.718345 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:26.718291    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:26.718400 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:26.718379    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:27.916016 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:27.915985    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:28.017462 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.017434    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:28.117172 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.117115    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:28.180905 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.180819    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:28.349885 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.349263    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:28.349885 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.349447    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:20:28.349885 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.349513    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:21:00.349494691 +0000 UTC m=+66.117217463 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 10 14:20:28.450462 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.450380    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:28.450627 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.450516    2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 10 14:20:28.450627 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.450590    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret podName:3779e8a4-b29e-4703-846e-1a6a3b58c4cd nodeName:}" failed. No retries permitted until 2026-09-10 14:21:00.450571923 +0000 UTC m=+66.218294686 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret") pod "global-pull-secret-syncer-4lk2v" (UID: "3779e8a4-b29e-4703-846e-1a6a3b58c4cd") : object "kube-system"/"original-pull-secret" not registered
Sep 10 14:20:28.551201 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.551159    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:28.551387 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.551323    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 10 14:20:28.551387 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.551348    2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 10 14:20:28.551387 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.551360    2578 projected.go:196] Error preparing data for projected volume kube-api-access-8nxcm for pod openshift-network-diagnostics/network-check-target-5hvxb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:20:28.551519 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.551427    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm podName:d1a9479e-dc19-4171-ac1b-f6e95fe6841c nodeName:}" failed. No retries permitted until 2026-09-10 14:21:00.551408297 +0000 UTC m=+66.319131059 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-8nxcm" (UniqueName: "kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm") pod "network-check-target-5hvxb" (UID: "d1a9479e-dc19-4171-ac1b-f6e95fe6841c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 10 14:20:28.721811 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.721721    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:28.721811 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.721774    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:28.721811 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:28.721774    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:28.722077 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.721861    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-4lk2v" podUID="3779e8a4-b29e-4703-846e-1a6a3b58c4cd"
Sep 10 14:20:28.722077 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.721952    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5hvxb" podUID="d1a9479e-dc19-4171-ac1b-f6e95fe6841c"
Sep 10 14:20:28.722077 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:28.722036    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:20:29.530106 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.530078    2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-68.ec2.internal" event="NodeReady"
Sep 10 14:20:29.530493 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.530276    2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 10 14:20:29.577249 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.577222    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kw77b"]
Sep 10 14:20:29.602860 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.602834    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ptbns"]
Sep 10 14:20:29.603020 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.602952    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.605448 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.605415    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 10 14:20:29.605591 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.605575    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-587sj\""
Sep 10 14:20:29.605929 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.605863    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 10 14:20:29.627061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.627027    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ptbns"]
Sep 10 14:20:29.627061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.627064    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kw77b"]
Sep 10 14:20:29.627222 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.627134    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ptbns"
Sep 10 14:20:29.629318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.629158    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-45wsb\""
Sep 10 14:20:29.629318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.629184    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 10 14:20:29.629318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.629195    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 10 14:20:29.629318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.629305    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 10 14:20:29.659818 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.659781    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78995de-d2c7-4b46-9118-e1187aba51e9-tmp-dir\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.659985 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.659824    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a78995de-d2c7-4b46-9118-e1187aba51e9-config-volume\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.659985 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.659872    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdcls\" (UniqueName: \"kubernetes.io/projected/a78995de-d2c7-4b46-9118-e1187aba51e9-kube-api-access-sdcls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.659985 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.659911    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.761204 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.761170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wppf7\" (UniqueName: \"kubernetes.io/projected/91b7b77a-784b-4ba5-9880-c54262d9c27c-kube-api-access-wppf7\") pod \"ingress-canary-ptbns\" (UID: \"91b7b77a-784b-4ba5-9880-c54262d9c27c\") " pod="openshift-ingress-canary/ingress-canary-ptbns"
Sep 10 14:20:29.761362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.761233    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78995de-d2c7-4b46-9118-e1187aba51e9-tmp-dir\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.761362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.761293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a78995de-d2c7-4b46-9118-e1187aba51e9-config-volume\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.761362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.761331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdcls\" (UniqueName: \"kubernetes.io/projected/a78995de-d2c7-4b46-9118-e1187aba51e9-kube-api-access-sdcls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.761469 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.761412    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91b7b77a-784b-4ba5-9880-c54262d9c27c-cert\") pod \"ingress-canary-ptbns\" (UID: \"91b7b77a-784b-4ba5-9880-c54262d9c27c\") " pod="openshift-ingress-canary/ingress-canary-ptbns"
Sep 10 14:20:29.761469 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.761445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.761534 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.761519    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a78995de-d2c7-4b46-9118-e1187aba51e9-tmp-dir\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.761619 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:29.761603    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:20:29.761692 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:29.761681    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:20:30.261663431 +0000 UTC m=+36.029386218 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:20:29.761939 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.761921    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a78995de-d2c7-4b46-9118-e1187aba51e9-config-volume\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.770613 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.770593    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdcls\" (UniqueName: \"kubernetes.io/projected/a78995de-d2c7-4b46-9118-e1187aba51e9-kube-api-access-sdcls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:29.862448 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.862411    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91b7b77a-784b-4ba5-9880-c54262d9c27c-cert\") pod \"ingress-canary-ptbns\" (UID: \"91b7b77a-784b-4ba5-9880-c54262d9c27c\") " pod="openshift-ingress-canary/ingress-canary-ptbns"
Sep 10 14:20:29.862616 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.862469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wppf7\" (UniqueName: \"kubernetes.io/projected/91b7b77a-784b-4ba5-9880-c54262d9c27c-kube-api-access-wppf7\") pod \"ingress-canary-ptbns\" (UID: \"91b7b77a-784b-4ba5-9880-c54262d9c27c\") " pod="openshift-ingress-canary/ingress-canary-ptbns"
Sep 10 14:20:29.864959 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.864932    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/91b7b77a-784b-4ba5-9880-c54262d9c27c-cert\") pod \"ingress-canary-ptbns\" (UID: \"91b7b77a-784b-4ba5-9880-c54262d9c27c\") " pod="openshift-ingress-canary/ingress-canary-ptbns"
Sep 10 14:20:29.869914 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.869893    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wppf7\" (UniqueName: \"kubernetes.io/projected/91b7b77a-784b-4ba5-9880-c54262d9c27c-kube-api-access-wppf7\") pod \"ingress-canary-ptbns\" (UID: \"91b7b77a-784b-4ba5-9880-c54262d9c27c\") " pod="openshift-ingress-canary/ingress-canary-ptbns"
Sep 10 14:20:29.937020 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.936980    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ptbns"
Sep 10 14:20:29.993279 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.993249    2578 generic.go:356] "Generic (PLEG): container finished" podID="1315f507-f0bd-46a5-afa4-749348c842e8" containerID="3298e9316edf9c019edf2d6b5ea0d75701a5d6b5d7f4bc004110a6f126a9b8bb" exitCode=0
Sep 10 14:20:29.993452 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:29.993295    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q24dz" event={"ID":"1315f507-f0bd-46a5-afa4-749348c842e8","Type":"ContainerDied","Data":"3298e9316edf9c019edf2d6b5ea0d75701a5d6b5d7f4bc004110a6f126a9b8bb"}
Sep 10 14:20:30.252530 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.252274    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ptbns"]
Sep 10 14:20:30.258007 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:20:30.257980    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b7b77a_784b_4ba5_9880_c54262d9c27c.slice/crio-56ba123a13d27c4a33c7b5b0b61606b3f85dfb5a720622a2579b090f4673f43a WatchSource:0}: Error finding container 56ba123a13d27c4a33c7b5b0b61606b3f85dfb5a720622a2579b090f4673f43a: Status 404 returned error can't find the container with id 56ba123a13d27c4a33c7b5b0b61606b3f85dfb5a720622a2579b090f4673f43a
Sep 10 14:20:30.266614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.266591    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:30.266722 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:30.266710    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:20:30.266795 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:30.266783    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:20:31.266766017 +0000 UTC m=+37.034488788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:20:30.718689 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.718650    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:20:30.719439 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.718700    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:20:30.719439 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.718650    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:20:30.721368 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.721348    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:20:30.721516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.721375    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:20:30.721516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.721381    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6szzq\""
Sep 10 14:20:30.721516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.721439    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:20:30.721516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.721381    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-flc9p\""
Sep 10 14:20:30.721516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.721499    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:20:30.999430 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.999338    2578 generic.go:356] "Generic (PLEG): container finished" podID="1315f507-f0bd-46a5-afa4-749348c842e8" containerID="3ea29199d51b88d72760afe4e00894f51cbae98850e3403dc12fbd35469a7fb4" exitCode=0
Sep 10 14:20:30.999603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:30.999449    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q24dz" event={"ID":"1315f507-f0bd-46a5-afa4-749348c842e8","Type":"ContainerDied","Data":"3ea29199d51b88d72760afe4e00894f51cbae98850e3403dc12fbd35469a7fb4"}
Sep 10 14:20:31.001095 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:31.001065    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ptbns" event={"ID":"91b7b77a-784b-4ba5-9880-c54262d9c27c","Type":"ContainerStarted","Data":"56ba123a13d27c4a33c7b5b0b61606b3f85dfb5a720622a2579b090f4673f43a"}
Sep 10 14:20:31.274802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:31.274687    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:31.274984 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:31.274881    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:20:31.274984 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:31.274958    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:20:33.274937943 +0000 UTC m=+39.042660705 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:20:32.007355 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:32.007318    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-q24dz" event={"ID":"1315f507-f0bd-46a5-afa4-749348c842e8","Type":"ContainerStarted","Data":"5995760fdee00e12cab82ecc4641f0dcc6bf89a3a299c1871eb21676a146384f"}
Sep 10 14:20:32.026606 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:32.026557    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-q24dz" podStartSLOduration=8.633849857 podStartE2EDuration="37.026540802s" totalImagesPullingTime="28.392690945s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:32.026148697 +0000 UTC m=+37.793871486" watchObservedRunningTime="2026-09-10 14:20:32.026540802 +0000 UTC m=+37.794263576"
Sep 10 14:20:32.147902 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:32.147867    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:32.245850 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:32.245815    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:33.011423 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:33.011386    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ptbns" event={"ID":"91b7b77a-784b-4ba5-9880-c54262d9c27c","Type":"ContainerStarted","Data":"f86d44615ac6deaccf7b4a5ed87e08d457bb26a1758c46e740816390cc32df18"}
Sep 10 14:20:33.029524 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:33.029472    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ptbns" podStartSLOduration=1.444300792 podStartE2EDuration="4.029456416s" totalImagesPullingTime="2.585155624s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:20:33.02885644 +0000 UTC m=+38.796579220" watchObservedRunningTime="2026-09-10 14:20:33.029456416 +0000 UTC m=+38.797179196"
Sep 10 14:20:33.291138 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:33.290889    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:33.291271 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:33.291064    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:20:33.291271 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:33.291216    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:20:37.291200436 +0000 UTC m=+43.058923194 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:20:34.400967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:34.400936    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:34.460830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:34.460795    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:37.316711 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:37.316669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:37.317144 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:37.316829    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:20:37.317144 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:37.316895    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:20:45.316878862 +0000 UTC m=+51.084601624 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:20:38.359676 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:38.359611    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:38.420065 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:38.420037    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:45.372118 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:45.372080    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:20:45.372582 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:45.372234    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:20:45.372582 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:20:45.372300    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:21:01.372282821 +0000 UTC m=+67.140005579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:20:47.950485 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:47.950450    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:48.014574 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:48.014545    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:49.065610 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:49.065571    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:49.129959 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:49.129931    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:54.987631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:54.987606    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wph9"
Sep 10 14:20:55.116890 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:55.116853    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:20:55.181816 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:20:55.181787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:00.187052 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.187018    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:00.371763 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.371717    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:21:00.373971 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.373947    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 10 14:21:00.382925 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:21:00.382905    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 14:21:00.382997 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:21:00.382990    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:22:04.382961001 +0000 UTC m=+130.150683759 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : secret "metrics-daemon-secret" not found
Sep 10 14:21:00.472137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.472068    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:21:00.474487 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.474465    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 10 14:21:00.485994 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.485967    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3779e8a4-b29e-4703-846e-1a6a3b58c4cd-original-pull-secret\") pod \"global-pull-secret-syncer-4lk2v\" (UID: \"3779e8a4-b29e-4703-846e-1a6a3b58c4cd\") " pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:21:00.572760 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.572722    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:21:00.574649 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.574632    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 10 14:21:00.585199 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.585180    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 10 14:21:00.596127 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.596108    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nxcm\" (UniqueName: \"kubernetes.io/projected/d1a9479e-dc19-4171-ac1b-f6e95fe6841c-kube-api-access-8nxcm\") pod \"network-check-target-5hvxb\" (UID: \"d1a9479e-dc19-4171-ac1b-f6e95fe6841c\") " pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:21:00.734660 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.734604    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-4lk2v"
Sep 10 14:21:00.741450 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.741416    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-6szzq\""
Sep 10 14:21:00.748837 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:00.748815    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:21:01.033205 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:01.033127    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-4lk2v"]
Sep 10 14:21:01.036675 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:21:01.036644    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3779e8a4_b29e_4703_846e_1a6a3b58c4cd.slice/crio-8d54f1f332313375017bb1a448c2d28866ed50f83a002174e94affa06bc6fe29 WatchSource:0}: Error finding container 8d54f1f332313375017bb1a448c2d28866ed50f83a002174e94affa06bc6fe29: Status 404 returned error can't find the container with id 8d54f1f332313375017bb1a448c2d28866ed50f83a002174e94affa06bc6fe29
Sep 10 14:21:01.064413 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:01.064381    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5hvxb"]
Sep 10 14:21:01.069157 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:21:01.069131    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1a9479e_dc19_4171_ac1b_f6e95fe6841c.slice/crio-11ccf96bb47d1144de744096a5659e177838654d00aebdac595f2bfdb6c6b732 WatchSource:0}: Error finding container 11ccf96bb47d1144de744096a5659e177838654d00aebdac595f2bfdb6c6b732: Status 404 returned error can't find the container with id 11ccf96bb47d1144de744096a5659e177838654d00aebdac595f2bfdb6c6b732
Sep 10 14:21:01.074250 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:01.074219    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5hvxb" event={"ID":"d1a9479e-dc19-4171-ac1b-f6e95fe6841c","Type":"ContainerStarted","Data":"11ccf96bb47d1144de744096a5659e177838654d00aebdac595f2bfdb6c6b732"}
Sep 10 14:21:01.075274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:01.075253    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lk2v" event={"ID":"3779e8a4-b29e-4703-846e-1a6a3b58c4cd","Type":"ContainerStarted","Data":"8d54f1f332313375017bb1a448c2d28866ed50f83a002174e94affa06bc6fe29"}
Sep 10 14:21:01.378674 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:01.378641    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:21:01.379106 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:21:01.378792    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:21:01.379106 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:21:01.378844    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:21:33.378829887 +0000 UTC m=+99.146552649 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:21:03.546718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:03.546646    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:03.636981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:03.636943    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:06.089957 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:06.089918    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5hvxb" event={"ID":"d1a9479e-dc19-4171-ac1b-f6e95fe6841c","Type":"ContainerStarted","Data":"db6593c46510e14d9242a9be74400bec28fed2ed74edb0d5464ee3a406787219"}
Sep 10 14:21:06.090425 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:06.090039    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:21:06.092489 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:06.092455    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-4lk2v" event={"ID":"3779e8a4-b29e-4703-846e-1a6a3b58c4cd","Type":"ContainerStarted","Data":"33b65d85fd9da2c3076b4bd86f227ec00eef73c48b6855b2ebc06854bae549b2"}
Sep 10 14:21:06.105698 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:06.105640    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5hvxb" podStartSLOduration=66.700533999 podStartE2EDuration="1m11.105623485s" totalImagesPullingTime="4.405089486s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:06.104280015 +0000 UTC m=+71.872002793" watchObservedRunningTime="2026-09-10 14:21:06.105623485 +0000 UTC m=+71.873346258"
Sep 10 14:21:06.121234 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:06.121177    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-4lk2v" podStartSLOduration=66.680604177 podStartE2EDuration="1m11.121158734s" totalImagesPullingTime="4.440554557s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:06.120002229 +0000 UTC m=+71.887725008" watchObservedRunningTime="2026-09-10 14:21:06.121158734 +0000 UTC m=+71.888881514"
Sep 10 14:21:17.445727 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.445600    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"]
Sep 10 14:21:17.449632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.449616    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.451322 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.451302    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 10 14:21:17.451768 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.451714    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 10 14:21:17.451768 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.451727    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 10 14:21:17.451768 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.451763    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 10 14:21:17.451949 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.451769    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 10 14:21:17.451949 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.451765    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 10 14:21:17.451949 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.451703    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 10 14:21:17.464377 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.464358    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"]
Sep 10 14:21:17.588441 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.588400    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.588441 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.588439    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/812e74c1-7ac0-4689-9d65-ab5356161bc2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.588672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.588475    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-ca\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.588672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.588573    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.588672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.588600    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rg6jb\" (UniqueName: \"kubernetes.io/projected/812e74c1-7ac0-4689-9d65-ab5356161bc2-kube-api-access-rg6jb\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.588672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.588634    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-hub\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.689069 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.689020    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.689069 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.689069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/812e74c1-7ac0-4689-9d65-ab5356161bc2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.689266 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.689101    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-ca\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.689266 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.689242    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.689334 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.689266    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rg6jb\" (UniqueName: \"kubernetes.io/projected/812e74c1-7ac0-4689-9d65-ab5356161bc2-kube-api-access-rg6jb\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.689334 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.689300    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-hub\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.689799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.689770    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/812e74c1-7ac0-4689-9d65-ab5356161bc2-additional-service-ca\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.692135 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.692106    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-ca\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.692251 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.692138    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.692251 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.692182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.692327 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.692249    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/812e74c1-7ac0-4689-9d65-ab5356161bc2-hub\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.696718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.696675    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rg6jb\" (UniqueName: \"kubernetes.io/projected/812e74c1-7ac0-4689-9d65-ab5356161bc2-kube-api-access-rg6jb\") pod \"cluster-proxy-proxy-agent-664c88b86b-57xkf\" (UID: \"812e74c1-7ac0-4689-9d65-ab5356161bc2\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:17.769765 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:17.769717    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:21:18.040608 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:18.040497    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"]
Sep 10 14:21:18.044181 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:21:18.044147    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod812e74c1_7ac0_4689_9d65_ab5356161bc2.slice/crio-06f9b403ad73f362e26210d21b949350fd44149829596d087bffa8a7c4b9ec44 WatchSource:0}: Error finding container 06f9b403ad73f362e26210d21b949350fd44149829596d087bffa8a7c4b9ec44: Status 404 returned error can't find the container with id 06f9b403ad73f362e26210d21b949350fd44149829596d087bffa8a7c4b9ec44
Sep 10 14:21:18.122211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:18.122179    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" event={"ID":"812e74c1-7ac0-4689-9d65-ab5356161bc2","Type":"ContainerStarted","Data":"06f9b403ad73f362e26210d21b949350fd44149829596d087bffa8a7c4b9ec44"}
Sep 10 14:21:22.136572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:22.136535    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" event={"ID":"812e74c1-7ac0-4689-9d65-ab5356161bc2","Type":"ContainerStarted","Data":"a76e469859f173de7604afb1f0be17f952cbefd3f55c4bbd6a87f30e92c84809"}
Sep 10 14:21:22.136957 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:22.136586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" event={"ID":"812e74c1-7ac0-4689-9d65-ab5356161bc2","Type":"ContainerStarted","Data":"d42e21761400d6a672dc402b30c8ecc0c9d166131ec47801fcd2c707d1b323a6"}
Sep 10 14:21:22.136957 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:22.136604    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" event={"ID":"812e74c1-7ac0-4689-9d65-ab5356161bc2","Type":"ContainerStarted","Data":"6a6601dc2820efa2bb8388ad8f8e3f8ccde8ca8442bbb852b8d86bf7302ffdd8"}
Sep 10 14:21:22.157955 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:22.157906    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" podStartSLOduration=1.544450077 podStartE2EDuration="5.157889707s" totalImagesPullingTime="3.61343963s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:21:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:21:22.157274624 +0000 UTC m=+87.924997404" watchObservedRunningTime="2026-09-10 14:21:22.157889707 +0000 UTC m=+87.925612515"
Sep 10 14:21:33.403262 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:33.403218    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:21:33.403677 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:21:33.403362    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:21:33.403677 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:21:33.403429    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:22:37.403413371 +0000 UTC m=+163.171136129 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:21:37.096576 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:37.096542    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5hvxb"
Sep 10 14:21:37.916420 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:37.916388    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:21:37.968995 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:21:37.968968    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:22:04.412988 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:22:04.412945    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:22:04.413466 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:22:04.413064    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 14:22:04.413466 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:22:04.413134    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:24:06.413120193 +0000 UTC m=+252.180842951 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : secret "metrics-daemon-secret" not found
Sep 10 14:22:32.613367 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:22:32.613325    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-kw77b" podUID="a78995de-d2c7-4b46-9118-e1187aba51e9"
Sep 10 14:22:33.337211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:22:33.337179    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kw77b"
Sep 10 14:22:33.728640 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:22:33.728536    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:22:37.436725 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:22:37.436669    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:22:37.437155 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:22:37.436850    2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 10 14:22:37.437155 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:22:37.436931    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls podName:a78995de-d2c7-4b46-9118-e1187aba51e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:24:39.436911216 +0000 UTC m=+285.204633992 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls") pod "dns-default-kw77b" (UID: "a78995de-d2c7-4b46-9118-e1187aba51e9") : secret "dns-default-metrics-tls" not found
Sep 10 14:22:44.718939 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:22:44.718902    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:23:45.765165 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:23:45.765132    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:06.444543 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:06.444491    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:24:06.445069 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:06.444643    2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 10 14:24:06.445069 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:06.444716    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs podName:cd204bcf-eade-4a4b-95c4-b7c1ed8504af nodeName:}" failed. No retries permitted until 2026-09-10 14:26:08.44469969 +0000 UTC m=+374.212422449 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs") pod "network-metrics-daemon-xcqxv" (UID: "cd204bcf-eade-4a4b-95c4-b7c1ed8504af") : secret "metrics-daemon-secret" not found
Sep 10 14:24:20.846159 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:20.846083    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:20.950066 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:20.950027    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:24.664782 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:24.664730    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:24.730369 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:24.730342    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:25.201350 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.201314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:25.293442 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.293379    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:25.393284 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.393241    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:25.456496 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.456415    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:25.661166 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.661134    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-65f2t"]
Sep 10 14:24:25.664004 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.663989    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.666557 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.666537    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 10 14:24:25.666886 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.666578    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 10 14:24:25.666886 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.666824    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 10 14:24:25.669614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.669598    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mxclh\""
Sep 10 14:24:25.670440 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.670422    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 10 14:24:25.680570 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.680551    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-65f2t"]
Sep 10 14:24:25.702962 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.702934    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:25.781289 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.781211    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.781289 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.781253    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96655b5e-c11e-46fa-b7f8-c68350542eaa-crio-socket\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.781289 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.781286    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.781547 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.781347    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96655b5e-c11e-46fa-b7f8-c68350542eaa-data-volume\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.781547 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.781381    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dlck\" (UniqueName: \"kubernetes.io/projected/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-api-access-9dlck\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.800508 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.800480    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:25.882023 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.881981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96655b5e-c11e-46fa-b7f8-c68350542eaa-crio-socket\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.882189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.882029    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.882189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.882056    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96655b5e-c11e-46fa-b7f8-c68350542eaa-data-volume\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.882189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.882076    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dlck\" (UniqueName: \"kubernetes.io/projected/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-api-access-9dlck\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.882189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.882118    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/96655b5e-c11e-46fa-b7f8-c68350542eaa-crio-socket\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.882189 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:25.882168    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:25.882420 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:25.882241    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:26.382220861 +0000 UTC m=+272.149943633 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:25.882420 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.882264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.882420 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:25.882361    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:24:25.882420 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.882413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/96655b5e-c11e-46fa-b7f8-c68350542eaa-data-volume\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:25.882573 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:25.882419    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:26.382406186 +0000 UTC m=+272.150128959 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : configmap "kube-rbac-proxy" not found
Sep 10 14:24:25.890996 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:25.890963    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dlck\" (UniqueName: \"kubernetes.io/projected/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-api-access-9dlck\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:26.076888 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:26.076859    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:26.134875 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:26.134824    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:26.385930 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:26.385839    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:26.386181 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:26.385931    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:26.386181 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:26.385993    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:26.386181 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:26.386012    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:24:26.386181 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:26.386057    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:27.386041669 +0000 UTC m=+273.153764428 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:26.386181 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:26.386071    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:27.386065848 +0000 UTC m=+273.153788607 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : configmap "kube-rbac-proxy" not found
Sep 10 14:24:27.393441 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:27.393399    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:27.393849 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:27.393452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:27.393849 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:27.393548    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:24:27.393849 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:27.393565    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:27.393849 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:27.393629    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:29.393615182 +0000 UTC m=+275.161337945 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:27.393849 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:27.393641    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:29.393635069 +0000 UTC m=+275.161357829 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : configmap "kube-rbac-proxy" not found
Sep 10 14:24:27.598398 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:27.598368    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:27.739586 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:27.739503    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:27.796733 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:27.796698    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:28.426568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:28.426540    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kk5vd_607b65d4-332c-465a-92af-3363fc98146b/dns-node-resolver/0.log"
Sep 10 14:24:28.669296 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:28.669264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:29.408286 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:29.408243    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:29.408485 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:29.408308    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:29.408485 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:29.408376    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:24:29.408485 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:29.408442    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:33.408426539 +0000 UTC m=+279.176149298 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : configmap "kube-rbac-proxy" not found
Sep 10 14:24:29.408485 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:29.408476    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:29.408671 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:29.408535    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:33.408522925 +0000 UTC m=+279.176245684 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:29.625506 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:29.625478    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cdx6z_ad4815b3-1e11-427b-86f1-f65145c8cf90/node-ca/0.log"
Sep 10 14:24:30.425856 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:30.425787    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ptbns_91b7b77a-784b-4ba5-9880-c54262d9c27c/serve-healthcheck-canary/0.log"
Sep 10 14:24:31.516415 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:31.516381    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:31.617885 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:31.617851    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:31.716615 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:31.716579    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:31.833712 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:31.833672    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:31.963952 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:31.963887    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:32.064391 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:32.064358    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:32.221960 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:32.221842    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:32.603184 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:32.603133    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:33.438332 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:33.438292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:33.438513 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:33.438357    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:33.438513 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:33.438427    2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 10 14:24:33.438513 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:33.438458    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:33.438513 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:33.438492    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:41.438476084 +0000 UTC m=+287.206198843 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : configmap "kube-rbac-proxy" not found
Sep 10 14:24:33.438513 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:33.438507    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:41.438500915 +0000 UTC m=+287.206223673 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:33.606316 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:33.606282    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:33.833978 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:33.833945    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.208911 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:34.208792    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.408513 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:34.408478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.812248 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:34.812217    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:34.991821 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:34.991792    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:35.403382 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:35.403349    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:35.505131 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:35.505098    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:35.613158 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:35.613130    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:36.011299 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:36.011262    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:36.207958 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:36.207916    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:36.338118 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:36.338088    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-kw77b" podUID="a78995de-d2c7-4b46-9118-e1187aba51e9"
Sep 10 14:24:36.406599 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:36.406565    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:36.685922 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:36.685843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kw77b"
Sep 10 14:24:36.809509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:36.809478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:37.411174 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:37.411146    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:37.810178 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:37.810099    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:37.938402 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:37.938371    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:38.039156 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:38.039125    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:38.379043 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:38.379008    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:38.436719 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:38.436687    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:39.006925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:39.006895    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:39.412850 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:39.412819    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:39.482115 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:39.482082    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:24:39.484508 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:39.484477    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a78995de-d2c7-4b46-9118-e1187aba51e9-metrics-tls\") pod \"dns-default-kw77b\" (UID: \"a78995de-d2c7-4b46-9118-e1187aba51e9\") " pod="openshift-dns/dns-default-kw77b"
Sep 10 14:24:39.688664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:39.688582    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-587sj\""
Sep 10 14:24:39.697281 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:39.697257    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kw77b"
Sep 10 14:24:39.807969 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:39.807937    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:39.966594 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:39.966521    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kw77b"]
Sep 10 14:24:39.969470 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:24:39.969439    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda78995de_d2c7_4b46_9118_e1187aba51e9.slice/crio-4abd234d5ea9e23de9ad716565057dc163094cd04292b4cbfa3e2566f331bfdc WatchSource:0}: Error finding container 4abd234d5ea9e23de9ad716565057dc163094cd04292b4cbfa3e2566f331bfdc: Status 404 returned error can't find the container with id 4abd234d5ea9e23de9ad716565057dc163094cd04292b4cbfa3e2566f331bfdc
Sep 10 14:24:40.204582 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:40.204550    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:40.604973 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:40.604942    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:40.698484 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:40.698450    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kw77b" event={"ID":"a78995de-d2c7-4b46-9118-e1187aba51e9","Type":"ContainerStarted","Data":"4abd234d5ea9e23de9ad716565057dc163094cd04292b4cbfa3e2566f331bfdc"}
Sep 10 14:24:40.814640 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:40.814604    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:41.210085 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:41.210045    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:41.497187 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:41.497101    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:41.497187 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:41.497166    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:41.497422 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:41.497291    2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:41.497422 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:41.497416    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls podName:96655b5e-c11e-46fa-b7f8-c68350542eaa nodeName:}" failed. No retries permitted until 2026-09-10 14:24:57.497392359 +0000 UTC m=+303.265115126 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls") pod "insights-runtime-extractor-65f2t" (UID: "96655b5e-c11e-46fa-b7f8-c68350542eaa") : secret "insights-runtime-extractor-tls" not found
Sep 10 14:24:41.497730 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:41.497701    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/96655b5e-c11e-46fa-b7f8-c68350542eaa-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:41.805226 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:41.805130    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:42.005466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:42.005425    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:42.212610 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:42.212567    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:42.705824 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:42.705790    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kw77b" event={"ID":"a78995de-d2c7-4b46-9118-e1187aba51e9","Type":"ContainerStarted","Data":"b1ffe7556ac945057331716ef106cfd5079c1a3b45a8b613e672cb54e53d8301"}
Sep 10 14:24:42.706014 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:42.705842    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kw77b" event={"ID":"a78995de-d2c7-4b46-9118-e1187aba51e9","Type":"ContainerStarted","Data":"cb22777d9e8b5341d6b959b6b0ea4de76830f1bbca217b1f3d7e0daff4a7c0c3"}
Sep 10 14:24:42.706014 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:42.705924    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kw77b"
Sep 10 14:24:42.725912 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:42.725826    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-kw77b" podStartSLOduration=251.270959841 podStartE2EDuration="4m13.725813153s" totalImagesPullingTime="2.454853312s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:24:42.724922867 +0000 UTC m=+288.492645648" watchObservedRunningTime="2026-09-10 14:24:42.725813153 +0000 UTC m=+288.493535933"
Sep 10 14:24:43.083979 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:43.083948    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:43.185877 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:43.185838    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:43.279564 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:43.279534    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:43.432422 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:43.432346    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:43.768335 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:43.768256    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:43.867009 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:43.866977    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:43.968309 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:43.968281    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:44.054233 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:44.054161    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:44.166381 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:44.166351    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:44.760850 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:44.760818    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:44.949145 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:44.949104    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:45.149394 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:45.149366    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:45.579718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:45.579689    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:45.681664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:45.681636    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:47.719395 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:24:47.719353    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-xcqxv" podUID="cd204bcf-eade-4a4b-95c4-b7c1ed8504af"
Sep 10 14:24:50.199657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.199628    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-8657cbd865-5nzzg"]
Sep 10 14:24:50.203003 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.202982    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.204977 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.204956    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 10 14:24:50.205494 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.205470    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 10 14:24:50.205590 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.205511    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-vzc7d\""
Sep 10 14:24:50.205648 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.205587    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 10 14:24:50.212034 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.211997    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 10 14:24:50.218808 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.218785    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8657cbd865-5nzzg"]
Sep 10 14:24:50.278504 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.278471    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:50.358682 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.358649    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-registry-tls\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.358682 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.358683    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffb246e9-d58f-439c-904b-07a3ce72d2ca-image-registry-private-configuration\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.358884 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.358702    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffb246e9-d58f-439c-904b-07a3ce72d2ca-ca-trust-extracted\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.358884 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.358721    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7m84h\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-kube-api-access-7m84h\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.358884 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.358854    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffb246e9-d58f-439c-904b-07a3ce72d2ca-installation-pull-secrets\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.358985 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.358886    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffb246e9-d58f-439c-904b-07a3ce72d2ca-registry-certificates\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.358985 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.358907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffb246e9-d58f-439c-904b-07a3ce72d2ca-trusted-ca\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.359047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.358982    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-bound-sa-token\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.459786 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.459683    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-registry-tls\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.459786 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.459716    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffb246e9-d58f-439c-904b-07a3ce72d2ca-image-registry-private-configuration\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.459786 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.459761    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffb246e9-d58f-439c-904b-07a3ce72d2ca-ca-trust-extracted\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.459786 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.459782    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7m84h\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-kube-api-access-7m84h\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.460082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.459826    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffb246e9-d58f-439c-904b-07a3ce72d2ca-installation-pull-secrets\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.460082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.459849    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffb246e9-d58f-439c-904b-07a3ce72d2ca-registry-certificates\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.460082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.459867    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffb246e9-d58f-439c-904b-07a3ce72d2ca-trusted-ca\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.460082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.459911    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-bound-sa-token\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.460279 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.460166    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ffb246e9-d58f-439c-904b-07a3ce72d2ca-ca-trust-extracted\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.460869 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.460844    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ffb246e9-d58f-439c-904b-07a3ce72d2ca-registry-certificates\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.460958 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.460934    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ffb246e9-d58f-439c-904b-07a3ce72d2ca-trusted-ca\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.462369 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.462349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ffb246e9-d58f-439c-904b-07a3ce72d2ca-installation-pull-secrets\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.462736 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.462711    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ffb246e9-d58f-439c-904b-07a3ce72d2ca-image-registry-private-configuration\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.462879 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.462861    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-registry-tls\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.473158 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.473132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7m84h\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-kube-api-access-7m84h\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.474188 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.474172    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ffb246e9-d58f-439c-904b-07a3ce72d2ca-bound-sa-token\") pod \"image-registry-8657cbd865-5nzzg\" (UID: \"ffb246e9-d58f-439c-904b-07a3ce72d2ca\") " pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.513703 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.513675    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:50.579816 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.579786    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:50.787331 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:50.787255    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-8657cbd865-5nzzg"]
Sep 10 14:24:50.790454 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:24:50.790427    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb246e9_d58f_439c_904b_07a3ce72d2ca.slice/crio-a339e4d32179fbb7bee9f1e32f193c695e4f1d6f2d1176781872a7655b2ecff1 WatchSource:0}: Error finding container a339e4d32179fbb7bee9f1e32f193c695e4f1d6f2d1176781872a7655b2ecff1: Status 404 returned error can't find the container with id a339e4d32179fbb7bee9f1e32f193c695e4f1d6f2d1176781872a7655b2ecff1
Sep 10 14:24:51.732092 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:51.732054    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8657cbd865-5nzzg" event={"ID":"ffb246e9-d58f-439c-904b-07a3ce72d2ca","Type":"ContainerStarted","Data":"067ab17c1ba8e65481bdcd677311f3d124ed9a39bbf865bc7d643872055137c8"}
Sep 10 14:24:51.732532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:51.732108    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-8657cbd865-5nzzg" event={"ID":"ffb246e9-d58f-439c-904b-07a3ce72d2ca","Type":"ContainerStarted","Data":"a339e4d32179fbb7bee9f1e32f193c695e4f1d6f2d1176781872a7655b2ecff1"}
Sep 10 14:24:51.732532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:51.732144    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:24:51.750271 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:51.750221    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-8657cbd865-5nzzg" podStartSLOduration=1.750203744 podStartE2EDuration="1.750203744s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:24:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:24:51.749109844 +0000 UTC m=+297.516832626" watchObservedRunningTime="2026-09-10 14:24:51.750203744 +0000 UTC m=+297.517926525"
Sep 10 14:24:52.710901 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:52.710870    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kw77b"
Sep 10 14:24:53.354942 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:53.354897    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:54.594455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:54.594429    2578 kubelet.go:1648] "Image garbage collection succeeded"
Sep 10 14:24:56.670945 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:56.670907    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:57.367667 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:57.367637    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:57.468075 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:57.468042    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:57.511741 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:57.511717    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:57.514229 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:57.514199    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/96655b5e-c11e-46fa-b7f8-c68350542eaa-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-65f2t\" (UID: \"96655b5e-c11e-46fa-b7f8-c68350542eaa\") " pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:57.774199 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:57.774125    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mxclh\""
Sep 10 14:24:57.783479 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:57.783457    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-65f2t"
Sep 10 14:24:58.054993 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:58.054874    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-65f2t"]
Sep 10 14:24:58.058530 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:24:58.058504    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96655b5e_c11e_46fa_b7f8_c68350542eaa.slice/crio-7ce001ef9e301da233c5cce05b6b5f8a6d3b70157376756686aed7927c251540 WatchSource:0}: Error finding container 7ce001ef9e301da233c5cce05b6b5f8a6d3b70157376756686aed7927c251540: Status 404 returned error can't find the container with id 7ce001ef9e301da233c5cce05b6b5f8a6d3b70157376756686aed7927c251540
Sep 10 14:24:58.114262 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:58.114242    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:24:58.755718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:58.755680    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-65f2t" event={"ID":"96655b5e-c11e-46fa-b7f8-c68350542eaa","Type":"ContainerStarted","Data":"b50b48c6abbfedee4455861de3a101496c8ebe2bb72208ab5e0b63a9317b2c92"}
Sep 10 14:24:58.755908 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:58.755736    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-65f2t" event={"ID":"96655b5e-c11e-46fa-b7f8-c68350542eaa","Type":"ContainerStarted","Data":"7ce001ef9e301da233c5cce05b6b5f8a6d3b70157376756686aed7927c251540"}
Sep 10 14:24:59.334762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.334717    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:59.396627 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.396598    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:59.460742 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.460703    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:59.763246 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.763169    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-65f2t" event={"ID":"96655b5e-c11e-46fa-b7f8-c68350542eaa","Type":"ContainerStarted","Data":"631f10d9db05e3bfcd48fd435ea8e8df922b88d49d522bef4e3e8ccea70dcc68"}
Sep 10 14:24:59.971845 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.971817    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7db4d96684-6g5tl"]
Sep 10 14:24:59.974705 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.974690    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:24:59.977639 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.977621    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 14:24:59.977823 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.977802    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 14:24:59.977908 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.977833    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zf5dh\""
Sep 10 14:24:59.977980 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.977961    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 14:24:59.978155 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.978133    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 14:24:59.978275 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.978183    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 14:24:59.978275 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.978208    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 14:24:59.978275 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.978237    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 14:24:59.985781 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.985730    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:24:59.990175 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:24:59.990155    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 14:25:00.006437 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.006403    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7db4d96684-6g5tl"]
Sep 10 14:25:00.034759 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.034654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-oauth-config\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.034759 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.034707    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-772m7\" (UniqueName: \"kubernetes.io/projected/80333348-3679-4c8f-85c4-476761959843-kube-api-access-772m7\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.034965 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.034766    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-trusted-ca-bundle\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.034965 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.034789    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-serving-cert\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.034965 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.034855    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-oauth-serving-cert\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.034965 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.034912    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-console-config\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.034965 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.034935    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-service-ca\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.065632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.065594    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:00.135659 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.135623    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-772m7\" (UniqueName: \"kubernetes.io/projected/80333348-3679-4c8f-85c4-476761959843-kube-api-access-772m7\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.135659 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.135659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-trusted-ca-bundle\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.135953 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.135676    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-serving-cert\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.135953 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.135699    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-oauth-serving-cert\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.135953 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.135739    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-console-config\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.135953 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.135791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-service-ca\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.135953 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.135851    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-oauth-config\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.136705 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.136647    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-oauth-serving-cert\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.136860 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.136706    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-console-config\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.136860 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.136831    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-trusted-ca-bundle\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.137245 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.137216    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-service-ca\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.138601 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.138580    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-oauth-config\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.138717 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.138658    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-serving-cert\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.145641 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.145618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-772m7\" (UniqueName: \"kubernetes.io/projected/80333348-3679-4c8f-85c4-476761959843-kube-api-access-772m7\") pod \"console-7db4d96684-6g5tl\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") " pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.284373 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.284336    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:00.718932 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.718902    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:25:00.728117 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.728092    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7db4d96684-6g5tl"]
Sep 10 14:25:00.731154 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:25:00.731130    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80333348_3679_4c8f_85c4_476761959843.slice/crio-dff0de71ca18c3262d8052cf71b0f83c2000a64a6f4cdc874bf5c981d2404a97 WatchSource:0}: Error finding container dff0de71ca18c3262d8052cf71b0f83c2000a64a6f4cdc874bf5c981d2404a97: Status 404 returned error can't find the container with id dff0de71ca18c3262d8052cf71b0f83c2000a64a6f4cdc874bf5c981d2404a97
Sep 10 14:25:00.768050 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.768015    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-65f2t" event={"ID":"96655b5e-c11e-46fa-b7f8-c68350542eaa","Type":"ContainerStarted","Data":"daa8f69de1ea199c28a56b4fe6f8b0a57217e4aa681075b958304d870b68fc59"}
Sep 10 14:25:00.769179 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.769155    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7db4d96684-6g5tl" event={"ID":"80333348-3679-4c8f-85c4-476761959843","Type":"ContainerStarted","Data":"dff0de71ca18c3262d8052cf71b0f83c2000a64a6f4cdc874bf5c981d2404a97"}
Sep 10 14:25:00.786471 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:00.786422    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-65f2t" podStartSLOduration=33.455067332 podStartE2EDuration="35.786407477s" totalImagesPullingTime="2.331340145s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 14:24:25 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:25:00.785996482 +0000 UTC m=+306.553719263" watchObservedRunningTime="2026-09-10 14:25:00.786407477 +0000 UTC m=+306.554130258"
Sep 10 14:25:01.921548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:01.921508    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:02.036298 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:02.036257    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:02.123924 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:02.123874    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:02.200942 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:02.200858    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:03.230387 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:03.230354    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:03.781336 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:03.781301    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7db4d96684-6g5tl" event={"ID":"80333348-3679-4c8f-85c4-476761959843","Type":"ContainerStarted","Data":"55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb"}
Sep 10 14:25:05.562379 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:05.562337    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:06.022673 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.022638    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:06.053704 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.053658    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7db4d96684-6g5tl" podStartSLOduration=4.508990251 podStartE2EDuration="7.053641228s" totalImagesPullingTime="2.544650977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:25:03.808418538 +0000 UTC m=+309.576141319" watchObservedRunningTime="2026-09-10 14:25:06.053641228 +0000 UTC m=+311.821364018"
Sep 10 14:25:06.054971 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.054947    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"]
Sep 10 14:25:06.058274 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.058259    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.063168 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.063145    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-jcrqz\""
Sep 10 14:25:06.063272 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.063173    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 10 14:25:06.065682 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.065663    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 10 14:25:06.069552 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.069530    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 10 14:25:06.069655 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.069558    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 10 14:25:06.069655 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.069559    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 10 14:25:06.069895 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.069882    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 10 14:25:06.070365 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.070350    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.070400 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.070384    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6j45\" (UniqueName: \"kubernetes.io/projected/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-api-access-d6j45\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.070434 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.070403    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.070467 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.070446    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.070500 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.070492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.070534 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.070524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.097996 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.097968    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"]
Sep 10 14:25:06.171427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.171393    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.171604 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.171444    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6j45\" (UniqueName: \"kubernetes.io/projected/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-api-access-d6j45\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.171604 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.171561    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.171728 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.171608    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.171728 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.171650    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.171728 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.171674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.172045 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.172021    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-volume-directive-shadow\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.172204 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.172183    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-metrics-client-ca\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.172259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.172237    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.174079 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.174057    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-tls\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.174166 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.174149    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.204543 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.204516    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6j45\" (UniqueName: \"kubernetes.io/projected/3ca48f0c-9fd1-4763-9f96-92cbdd07510e-kube-api-access-d6j45\") pod \"kube-state-metrics-74cb57df7-v4mlz\" (UID: \"3ca48f0c-9fd1-4763-9f96-92cbdd07510e\") " pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.261465 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.261435    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-pcxg8"]
Sep 10 14:25:06.264778 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.264740    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.269335 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.269313    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 10 14:25:06.269538 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.269518    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:06.271347 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.271328    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4vtq9\""
Sep 10 14:25:06.271938 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.271916    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 10 14:25:06.272032 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-root\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.272099 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272056    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-wtmp\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.272146 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272109    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-textfile\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.272146 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272138    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-accelerators-collector-config\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.272216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272156    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-sys\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.272216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58s68\" (UniqueName: \"kubernetes.io/projected/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-kube-api-access-58s68\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.272216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272196    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-metrics-client-ca\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.272306 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272233    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.272306 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.272274    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-tls\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.277026 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.276974    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 10 14:25:06.366599 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.366564    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"
Sep 10 14:25:06.372891 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.372864    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-textfile\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.372991 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.372902    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-accelerators-collector-config\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.372991 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.372933    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-sys\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.372991 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.372961    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58s68\" (UniqueName: \"kubernetes.io/projected/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-kube-api-access-58s68\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373136 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.372999    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-metrics-client-ca\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373136 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373022    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373136 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373030    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-sys\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373136 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373063    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-tls\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373136 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373090    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-root\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373136 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-wtmp\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373195    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-textfile\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373260    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-wtmp\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373561 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373471    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-accelerators-collector-config\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373585    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-metrics-client-ca\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.373670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.373644    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-root\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.375921 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.375896    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.376019 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.375913    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-node-exporter-tls\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.401736 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.401709    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-58s68\" (UniqueName: \"kubernetes.io/projected/ee6f2e2e-5eab-468d-8bbf-de67d03f1152-kube-api-access-58s68\") pod \"node-exporter-pcxg8\" (UID: \"ee6f2e2e-5eab-468d-8bbf-de67d03f1152\") " pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.542081 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.541999    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:06.574583 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.574288    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-pcxg8"
Sep 10 14:25:06.586635 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:25:06.586604    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6f2e2e_5eab_468d_8bbf_de67d03f1152.slice/crio-4905dd6df5e63cddbc3b5d8fa947d4b59d04fc04538096d0d16ed937b1b2d72a WatchSource:0}: Error finding container 4905dd6df5e63cddbc3b5d8fa947d4b59d04fc04538096d0d16ed937b1b2d72a: Status 404 returned error can't find the container with id 4905dd6df5e63cddbc3b5d8fa947d4b59d04fc04538096d0d16ed937b1b2d72a
Sep 10 14:25:06.692349 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.692313    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:06.699072 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.699046    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz"]
Sep 10 14:25:06.702309 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:25:06.702286    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ca48f0c_9fd1_4763_9f96_92cbdd07510e.slice/crio-b41ca3866d6c55a6d1eaa3959a686c763c3220fae5bcdd0f316b4178f842f6dc WatchSource:0}: Error finding container b41ca3866d6c55a6d1eaa3959a686c763c3220fae5bcdd0f316b4178f842f6dc: Status 404 returned error can't find the container with id b41ca3866d6c55a6d1eaa3959a686c763c3220fae5bcdd0f316b4178f842f6dc
Sep 10 14:25:06.792820 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.792722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz" event={"ID":"3ca48f0c-9fd1-4763-9f96-92cbdd07510e","Type":"ContainerStarted","Data":"b41ca3866d6c55a6d1eaa3959a686c763c3220fae5bcdd0f316b4178f842f6dc"}
Sep 10 14:25:06.793811 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.793788    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pcxg8" event={"ID":"ee6f2e2e-5eab-468d-8bbf-de67d03f1152","Type":"ContainerStarted","Data":"4905dd6df5e63cddbc3b5d8fa947d4b59d04fc04538096d0d16ed937b1b2d72a"}
Sep 10 14:25:06.917263 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:06.917225    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:07.099477 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.099447    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:07.411672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.411535    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:25:07.415688 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.415661    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.423214 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423187    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 14:25:07.423671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423431    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 14:25:07.423671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423482    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 14:25:07.423671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423431    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 14:25:07.423671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423569    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 14:25:07.423671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423642    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 14:25:07.424185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423729    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7xdxb\""
Sep 10 14:25:07.424185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423778    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 14:25:07.424185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423844    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 14:25:07.424185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.423927    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 14:25:07.455037 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.455007    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:25:07.480582 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480549    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtccj\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-kube-api-access-gtccj\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.480777 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480598    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.480777 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480625    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.480777 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480653    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.480777 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480704    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-volume\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.480777 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480739    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.481033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480863    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.481033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480918    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-out\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.481033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480944    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.481033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.480977    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.481033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.481017    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.481269 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.481055    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.481269 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.481143    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-web-config\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.581531 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581495    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581619    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-web-config\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581625    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtccj\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-kube-api-access-gtccj\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581905    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581934    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581957    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-volume\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.581980    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.582073    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.582129    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-out\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.582153    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.582215    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.582310    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.582455 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:25:07.582345    2578 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
Sep 10 14:25:07.582455 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:25:07.582411    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls podName:c925a7e5-4e7d-49c2-a89c-cbf2e973fb60 nodeName:}" failed. No retries permitted until 2026-09-10 14:25:08.082393887 +0000 UTC m=+313.850116649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60") : secret "alertmanager-main-tls" not found
Sep 10 14:25:07.582815 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.582576    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.583375 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.582978    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.586103 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.586058    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-out\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.587265 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.586804    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.587265 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.587052    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.587265 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.587087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.587933 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.587876    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.588266 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.588246    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.589084 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.589044    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-volume\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.589084 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.589062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-web-config\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.595005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.594978    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtccj\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-kube-api-access-gtccj\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:07.858400 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:07.858348    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:08.055506 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.055470    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:08.087045 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.087015    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:08.089906 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.089875    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:08.329030 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.329005    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:25:08.675787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.675730    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:25:08.692200 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:25:08.692168    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc925a7e5_4e7d_49c2_a89c_cbf2e973fb60.slice/crio-0827923f74a1a179b01b2afeab160e0700ceac5f54027ba6584721b7c7bf54fc WatchSource:0}: Error finding container 0827923f74a1a179b01b2afeab160e0700ceac5f54027ba6584721b7c7bf54fc: Status 404 returned error can't find the container with id 0827923f74a1a179b01b2afeab160e0700ceac5f54027ba6584721b7c7bf54fc
Sep 10 14:25:08.803119 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.803029    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerStarted","Data":"0827923f74a1a179b01b2afeab160e0700ceac5f54027ba6584721b7c7bf54fc"}
Sep 10 14:25:08.805427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.805398    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz" event={"ID":"3ca48f0c-9fd1-4763-9f96-92cbdd07510e","Type":"ContainerStarted","Data":"1420db01e4e7b9af2c6fb9bc77f3ef495468658696e5a6acfb664b1663c4e0f3"}
Sep 10 14:25:08.805556 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.805452    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz" event={"ID":"3ca48f0c-9fd1-4763-9f96-92cbdd07510e","Type":"ContainerStarted","Data":"2deaf3f3d90ac83c6c3c4eeb45a5b0cb3ef251d9ce9a1a5d4a20997eb6d52a0d"}
Sep 10 14:25:08.805556 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.805480    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz" event={"ID":"3ca48f0c-9fd1-4763-9f96-92cbdd07510e","Type":"ContainerStarted","Data":"43f971ec588c5c4ec5df646dd193aedecdd9c1c3a772fbb4e68ff83ba27d4f04"}
Sep 10 14:25:08.807144 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.807120    2578 generic.go:356] "Generic (PLEG): container finished" podID="ee6f2e2e-5eab-468d-8bbf-de67d03f1152" containerID="5b5ad9d246f9af9f74368975824c2c9ff7e88762e27e65ddd717a862d52a55e7" exitCode=0
Sep 10 14:25:08.807248 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.807153    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pcxg8" event={"ID":"ee6f2e2e-5eab-468d-8bbf-de67d03f1152","Type":"ContainerDied","Data":"5b5ad9d246f9af9f74368975824c2c9ff7e88762e27e65ddd717a862d52a55e7"}
Sep 10 14:25:08.857202 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:08.857140    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-74cb57df7-v4mlz" podStartSLOduration=2.272401339 podStartE2EDuration="3.857120424s" totalImagesPullingTime="1.584719085s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:25:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:25:08.850467895 +0000 UTC m=+314.618190677" watchObservedRunningTime="2026-09-10 14:25:08.857120424 +0000 UTC m=+314.624843210"
Sep 10 14:25:09.033402 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.033373    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:09.136235 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.136205    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:09.231283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.231248    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:09.331144 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.331099    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:09.457572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.457479    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:09.681948 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.681899    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:09.813292 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.813217    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pcxg8" event={"ID":"ee6f2e2e-5eab-468d-8bbf-de67d03f1152","Type":"ContainerStarted","Data":"1c3f1bcb9a8f1bde5df69a95cded0972ef4341def08a33c69d34489209b3cbc8"}
Sep 10 14:25:09.813432 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.813292    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-pcxg8" event={"ID":"ee6f2e2e-5eab-468d-8bbf-de67d03f1152","Type":"ContainerStarted","Data":"fc453ab5db67c8f8d7f3a79c053c443ce8a02ba044f97a27d82078fe134cad5b"}
Sep 10 14:25:09.849986 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:09.849931    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-pcxg8" podStartSLOduration=2.056807514 podStartE2EDuration="3.849909461s" totalImagesPullingTime="1.698365292s" totalInitContainerRuntime="94.736655ms" isStatefulPod=true podCreationTimestamp="2026-09-10 14:25:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:25:09.849773361 +0000 UTC m=+315.617496145" watchObservedRunningTime="2026-09-10 14:25:09.849909461 +0000 UTC m=+315.617632243"
Sep 10 14:25:10.285227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.285185    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:10.285481 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.285443    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:10.290169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.290150    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:10.717697 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.717660    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7db4d96684-6g5tl"]
Sep 10 14:25:10.801803 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.801772    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:10.818851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.818818    2578 generic.go:356] "Generic (PLEG): container finished" podID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerID="cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be" exitCode=0
Sep 10 14:25:10.819016 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.818909    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerDied","Data":"cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be"}
Sep 10 14:25:10.823116 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.823096    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:10.901298 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.901264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:10.933373 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.933336    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6875cf7786-5fvhn"]
Sep 10 14:25:10.936562 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.936538    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:10.939558 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.939525    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 10 14:25:10.939718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.939649    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-clgxj\""
Sep 10 14:25:10.939959 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.939941    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 10 14:25:10.940060 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.940028    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7ccfqr6qp51pb\""
Sep 10 14:25:10.940233 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.940215    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 10 14:25:10.941686 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.941660    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 10 14:25:10.949917 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:10.949898    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6875cf7786-5fvhn"]
Sep 10 14:25:11.013382 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.013290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e248996f-5cc7-4ede-9962-849e03af7eee-metrics-server-audit-profiles\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.013382 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.013342    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-secret-metrics-server-tls\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.013382 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.013374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e248996f-5cc7-4ede-9962-849e03af7eee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.013587 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.013429    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e248996f-5cc7-4ede-9962-849e03af7eee-audit-log\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.013587 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.013488    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-client-ca-bundle\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.013587 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.013517    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-secret-metrics-server-client-certs\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.013587 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.013535    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2rk5\" (UniqueName: \"kubernetes.io/projected/e248996f-5cc7-4ede-9962-849e03af7eee-kube-api-access-b2rk5\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.114712 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.114672    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e248996f-5cc7-4ede-9962-849e03af7eee-audit-log\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.114712 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.114717    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-client-ca-bundle\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.114967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.114736    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-secret-metrics-server-client-certs\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.114967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.114785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2rk5\" (UniqueName: \"kubernetes.io/projected/e248996f-5cc7-4ede-9962-849e03af7eee-kube-api-access-b2rk5\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.114967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.114816    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e248996f-5cc7-4ede-9962-849e03af7eee-metrics-server-audit-profiles\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.114967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.114854    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-secret-metrics-server-tls\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.115181 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.115003    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e248996f-5cc7-4ede-9962-849e03af7eee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.115181 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.115142    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/e248996f-5cc7-4ede-9962-849e03af7eee-audit-log\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.115736 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.115706    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e248996f-5cc7-4ede-9962-849e03af7eee-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.116058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.116040    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/e248996f-5cc7-4ede-9962-849e03af7eee-metrics-server-audit-profiles\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.117376 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.117354    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-client-ca-bundle\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.117560 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.117540    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-secret-metrics-server-client-certs\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.117603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.117543    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/e248996f-5cc7-4ede-9962-849e03af7eee-secret-metrics-server-tls\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.122064 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.122045    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2rk5\" (UniqueName: \"kubernetes.io/projected/e248996f-5cc7-4ede-9962-849e03af7eee-kube-api-access-b2rk5\") pod \"metrics-server-6875cf7786-5fvhn\" (UID: \"e248996f-5cc7-4ede-9962-849e03af7eee\") " pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.248487 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.248447    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:11.526871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.526843    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6875cf7786-5fvhn"]
Sep 10 14:25:11.529832 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:25:11.529798    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode248996f_5cc7_4ede_9962_849e03af7eee.slice/crio-ca0f1ef77a022bed2bc67a18cd3fb74fcb20d57c94bb8cc4871184064891cad8 WatchSource:0}: Error finding container ca0f1ef77a022bed2bc67a18cd3fb74fcb20d57c94bb8cc4871184064891cad8: Status 404 returned error can't find the container with id ca0f1ef77a022bed2bc67a18cd3fb74fcb20d57c94bb8cc4871184064891cad8
Sep 10 14:25:11.823388 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:11.823344    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn" event={"ID":"e248996f-5cc7-4ede-9962-849e03af7eee","Type":"ContainerStarted","Data":"ca0f1ef77a022bed2bc67a18cd3fb74fcb20d57c94bb8cc4871184064891cad8"}
Sep 10 14:25:12.256285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:12.256202    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:12.357664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:12.357630    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:12.529943 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:12.529864    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:12.742411 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:12.742379    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-8657cbd865-5nzzg"
Sep 10 14:25:12.922410 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:12.922380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:13.025347 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:13.025277    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:13.121961 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:13.121914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:13.833803 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:13.833765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerStarted","Data":"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a"}
Sep 10 14:25:13.835685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:13.835594    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn" event={"ID":"e248996f-5cc7-4ede-9962-849e03af7eee","Type":"ContainerStarted","Data":"eaaa9dbd184cf41bbabc097435604f5b822facf961fb8c82d1c23325c23d69ef"}
Sep 10 14:25:13.866104 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:13.865649    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn" podStartSLOduration=1.6812194969999998 podStartE2EDuration="3.865628374s" totalImagesPullingTime="2.184408877s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:25:13.864546547 +0000 UTC m=+319.632269330" watchObservedRunningTime="2026-09-10 14:25:13.865628374 +0000 UTC m=+319.633351156"
Sep 10 14:25:14.843027 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:14.842993    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerStarted","Data":"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b"}
Sep 10 14:25:14.843324 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:14.843062    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerStarted","Data":"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692"}
Sep 10 14:25:14.843324 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:14.843090    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerStarted","Data":"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379"}
Sep 10 14:25:14.843324 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:14.843114    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerStarted","Data":"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696"}
Sep 10 14:25:15.849736 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:15.849697    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerStarted","Data":"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33"}
Sep 10 14:25:15.878081 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:15.878010    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.044727551 podStartE2EDuration="8.877990103s" totalImagesPullingTime="4.815675339s" totalInitContainerRuntime="17.587213ms" isStatefulPod=false podCreationTimestamp="2026-09-10 14:25:07 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:25:15.876810526 +0000 UTC m=+321.644533311" watchObservedRunningTime="2026-09-10 14:25:15.877990103 +0000 UTC m=+321.645712884"
Sep 10 14:25:16.116521 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:16.116430    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:16.171001 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:16.170973    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:16.710006 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:16.709968    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:16.764907 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:16.764875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:17.770880 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:17.770824    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" podUID="812e74c1-7ac0-4689-9d65-ab5356161bc2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 14:25:23.363173 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:23.363131    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:23.482426 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:23.482377    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:24.112359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:24.112324    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:24.213211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:24.213177    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:26.885531 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:26.885487    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:26.983668 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:26.983614    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:27.083781 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:27.083732    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:27.771819 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:27.771779    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" podUID="812e74c1-7ac0-4689-9d65-ab5356161bc2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 14:25:31.248675 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:31.248638    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:31.249107 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:31.248704    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:36.843359 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:36.843306    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7db4d96684-6g5tl" podUID="80333348-3679-4c8f-85c4-476761959843" containerName="console" containerID="cri-o://55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb" gracePeriod=15
Sep 10 14:25:37.086002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.085978    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:37.230663 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.230570    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-trusted-ca-bundle\") pod \"80333348-3679-4c8f-85c4-476761959843\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") "
Sep 10 14:25:37.230663 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.230622    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-oauth-serving-cert\") pod \"80333348-3679-4c8f-85c4-476761959843\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") "
Sep 10 14:25:37.230663 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.230645    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-serving-cert\") pod \"80333348-3679-4c8f-85c4-476761959843\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") "
Sep 10 14:25:37.230663 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.230664    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-console-config\") pod \"80333348-3679-4c8f-85c4-476761959843\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") "
Sep 10 14:25:37.231013 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.230686    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-service-ca\") pod \"80333348-3679-4c8f-85c4-476761959843\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") "
Sep 10 14:25:37.231013 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.230711    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-772m7\" (UniqueName: \"kubernetes.io/projected/80333348-3679-4c8f-85c4-476761959843-kube-api-access-772m7\") pod \"80333348-3679-4c8f-85c4-476761959843\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") "
Sep 10 14:25:37.231013 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.230738    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-oauth-config\") pod \"80333348-3679-4c8f-85c4-476761959843\" (UID: \"80333348-3679-4c8f-85c4-476761959843\") "
Sep 10 14:25:37.231221 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.231187    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "80333348-3679-4c8f-85c4-476761959843" (UID: "80333348-3679-4c8f-85c4-476761959843"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:25:37.231424 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.231216    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-console-config" (OuterVolumeSpecName: "console-config") pod "80333348-3679-4c8f-85c4-476761959843" (UID: "80333348-3679-4c8f-85c4-476761959843"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:25:37.231424 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.231253    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "80333348-3679-4c8f-85c4-476761959843" (UID: "80333348-3679-4c8f-85c4-476761959843"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:25:37.231529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.231292    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-service-ca" (OuterVolumeSpecName: "service-ca") pod "80333348-3679-4c8f-85c4-476761959843" (UID: "80333348-3679-4c8f-85c4-476761959843"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:25:37.233225 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.233202    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "80333348-3679-4c8f-85c4-476761959843" (UID: "80333348-3679-4c8f-85c4-476761959843"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:25:37.233311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.233204    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80333348-3679-4c8f-85c4-476761959843-kube-api-access-772m7" (OuterVolumeSpecName: "kube-api-access-772m7") pod "80333348-3679-4c8f-85c4-476761959843" (UID: "80333348-3679-4c8f-85c4-476761959843"). InnerVolumeSpecName "kube-api-access-772m7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:25:37.233311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.233236    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "80333348-3679-4c8f-85c4-476761959843" (UID: "80333348-3679-4c8f-85c4-476761959843"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:25:37.331578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.331542    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-trusted-ca-bundle\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:25:37.331578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.331573    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-oauth-serving-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:25:37.331578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.331582    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-serving-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:25:37.331830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.331592    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-console-config\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:25:37.331830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.331601    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/80333348-3679-4c8f-85c4-476761959843-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:25:37.331830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.331608    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-772m7\" (UniqueName: \"kubernetes.io/projected/80333348-3679-4c8f-85c4-476761959843-kube-api-access-772m7\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:25:37.331830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.331616    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/80333348-3679-4c8f-85c4-476761959843-console-oauth-config\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:25:37.771204 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.771164    2578 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" podUID="812e74c1-7ac0-4689-9d65-ab5356161bc2" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500"
Sep 10 14:25:37.771376 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.771237    2578 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf"
Sep 10 14:25:37.771733 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.771695    2578 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"a76e469859f173de7604afb1f0be17f952cbefd3f55c4bbd6a87f30e92c84809"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" containerMessage="Container service-proxy failed liveness probe, will be restarted"
Sep 10 14:25:37.771790 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.771777    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" podUID="812e74c1-7ac0-4689-9d65-ab5356161bc2" containerName="service-proxy" containerID="cri-o://a76e469859f173de7604afb1f0be17f952cbefd3f55c4bbd6a87f30e92c84809" gracePeriod=30
Sep 10 14:25:37.928343 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.928295    2578 generic.go:356] "Generic (PLEG): container finished" podID="812e74c1-7ac0-4689-9d65-ab5356161bc2" containerID="a76e469859f173de7604afb1f0be17f952cbefd3f55c4bbd6a87f30e92c84809" exitCode=2
Sep 10 14:25:37.928724 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.928354    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" event={"ID":"812e74c1-7ac0-4689-9d65-ab5356161bc2","Type":"ContainerDied","Data":"a76e469859f173de7604afb1f0be17f952cbefd3f55c4bbd6a87f30e92c84809"}
Sep 10 14:25:37.929698 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.929678    2578 generic.go:356] "Generic (PLEG): container finished" podID="80333348-3679-4c8f-85c4-476761959843" containerID="55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb" exitCode=0
Sep 10 14:25:37.929801 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.929722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7db4d96684-6g5tl" event={"ID":"80333348-3679-4c8f-85c4-476761959843","Type":"ContainerDied","Data":"55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb"}
Sep 10 14:25:37.929801 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.929740    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7db4d96684-6g5tl"
Sep 10 14:25:37.929801 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.929781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7db4d96684-6g5tl" event={"ID":"80333348-3679-4c8f-85c4-476761959843","Type":"ContainerDied","Data":"dff0de71ca18c3262d8052cf71b0f83c2000a64a6f4cdc874bf5c981d2404a97"}
Sep 10 14:25:37.929913 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.929802    2578 scope.go:117] "RemoveContainer" containerID="55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb"
Sep 10 14:25:37.945358 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.945342    2578 scope.go:117] "RemoveContainer" containerID="55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb"
Sep 10 14:25:37.945649 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:25:37.945629    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb\": container with ID starting with 55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb not found: ID does not exist" containerID="55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb"
Sep 10 14:25:37.945709 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.945661    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb"} err="failed to get container status \"55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb\": rpc error: code = NotFound desc = could not find container \"55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb\": container with ID starting with 55890af02b10346163485d98d389a793f6aacc050d1bf5810a716504220d42eb not found: ID does not exist"
Sep 10 14:25:37.950810 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.950773    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7db4d96684-6g5tl"]
Sep 10 14:25:37.954135 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:37.954117    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7db4d96684-6g5tl"]
Sep 10 14:25:38.359910 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:38.359874    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:38.416654 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:38.416604    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:38.722761 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:38.722668    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80333348-3679-4c8f-85c4-476761959843" path="/var/lib/kubelet/pods/80333348-3679-4c8f-85c4-476761959843/volumes"
Sep 10 14:25:38.936091 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:38.936056    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-664c88b86b-57xkf" event={"ID":"812e74c1-7ac0-4689-9d65-ab5356161bc2","Type":"ContainerStarted","Data":"a57e69d68301b7ada981fe0f3a195617793bec10200aaeab1ed40c1a26a49ebd"}
Sep 10 14:25:51.254761 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:51.254675    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:51.258556 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:51.258532    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6875cf7786-5fvhn"
Sep 10 14:25:53.356621 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:53.356567    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:53.412572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:53.412536    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:25:55.677085 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:55.677057    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kw77b_a78995de-d2c7-4b46-9118-e1187aba51e9/dns/0.log"
Sep 10 14:25:55.683242 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:55.683225    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kw77b_a78995de-d2c7-4b46-9118-e1187aba51e9/kube-rbac-proxy/0.log"
Sep 10 14:25:56.044013 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:25:56.043942    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kk5vd_607b65d4-332c-465a-92af-3363fc98146b/dns-node-resolver/0.log"
Sep 10 14:26:03.035292 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:03.035258    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:03.094494 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:03.094457    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:04.089860 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:04.089827    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:04.187216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:04.187179    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:08.487047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:08.487014    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:26:08.489540 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:08.489507    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd204bcf-eade-4a4b-95c4-b7c1ed8504af-metrics-certs\") pod \"network-metrics-daemon-xcqxv\" (UID: \"cd204bcf-eade-4a4b-95c4-b7c1ed8504af\") " pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:26:08.521785 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:08.521512    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-flc9p\""
Sep 10 14:26:08.530358 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:08.530325    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xcqxv"
Sep 10 14:26:08.818295 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:08.816843    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xcqxv"]
Sep 10 14:26:08.822328 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:26:08.822284    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd204bcf_eade_4a4b_95c4_b7c1ed8504af.slice/crio-03d570bb7facff7a40e0e75a6aa6e9203d132b74a5942790c98f64535ddbfeb9 WatchSource:0}: Error finding container 03d570bb7facff7a40e0e75a6aa6e9203d132b74a5942790c98f64535ddbfeb9: Status 404 returned error can't find the container with id 03d570bb7facff7a40e0e75a6aa6e9203d132b74a5942790c98f64535ddbfeb9
Sep 10 14:26:09.044648 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:09.044612    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xcqxv" event={"ID":"cd204bcf-eade-4a4b-95c4-b7c1ed8504af","Type":"ContainerStarted","Data":"03d570bb7facff7a40e0e75a6aa6e9203d132b74a5942790c98f64535ddbfeb9"}
Sep 10 14:26:11.054835 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:11.054799    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xcqxv" event={"ID":"cd204bcf-eade-4a4b-95c4-b7c1ed8504af","Type":"ContainerStarted","Data":"811b54602f8fcfbf2267820560db6c541bf3a689fdab3f8dd4f96e873752519b"}
Sep 10 14:26:11.055340 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:11.054853    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xcqxv" event={"ID":"cd204bcf-eade-4a4b-95c4-b7c1ed8504af","Type":"ContainerStarted","Data":"b1bd57c40633401c0528ff514de4efc61d03d6b50eb9f40e9cc308c0f3a81b63"}
Sep 10 14:26:11.069653 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:11.069581    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xcqxv" podStartSLOduration=374.770413369 podStartE2EDuration="6m16.069561779s" totalImagesPullingTime="1.29914841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:11.0692335 +0000 UTC m=+376.836956282" watchObservedRunningTime="2026-09-10 14:26:11.069561779 +0000 UTC m=+376.837284559"
Sep 10 14:26:11.175295 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:11.175262    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:11.237289 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:11.237252    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:22.396952 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:22.396914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:22.498870 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:22.498834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:26.042510 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.042474    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c9cc5d77f-6dw7k"]
Sep 10 14:26:26.042927 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.042769    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80333348-3679-4c8f-85c4-476761959843" containerName="console"
Sep 10 14:26:26.042927 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.042780    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="80333348-3679-4c8f-85c4-476761959843" containerName="console"
Sep 10 14:26:26.042927 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.042830    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="80333348-3679-4c8f-85c4-476761959843" containerName="console"
Sep 10 14:26:26.047068 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.047048    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.050337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.050311    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 10 14:26:26.050737 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.050721    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 10 14:26:26.050804 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.050738    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 10 14:26:26.050804 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.050724    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 10 14:26:26.051043 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.051024    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-zf5dh\""
Sep 10 14:26:26.051661 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.051643    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 10 14:26:26.051967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.051950    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 10 14:26:26.054471 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.054453    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 10 14:26:26.055935 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.055913    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 10 14:26:26.069599 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.069572    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9cc5d77f-6dw7k"]
Sep 10 14:26:26.115497 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.115462    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:26.121581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.121554    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-trusted-ca-bundle\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.121678 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.121585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kf86s\" (UniqueName: \"kubernetes.io/projected/384e61b4-a5e3-43d6-8210-aa920b896545-kube-api-access-kf86s\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.121678 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.121614    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-service-ca\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.121678 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.121641    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-oauth-config\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.121849 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.121682    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-serving-cert\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.121849 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.121717    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-oauth-serving-cert\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.121849 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.121804    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-console-config\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.219118 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.219084    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:26.222220 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.222199    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-oauth-serving-cert\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.222291 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.222238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-console-config\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.222291 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.222269    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-trusted-ca-bundle\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.222291 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.222286    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kf86s\" (UniqueName: \"kubernetes.io/projected/384e61b4-a5e3-43d6-8210-aa920b896545-kube-api-access-kf86s\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.222435 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.222315    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-service-ca\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.222435 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.222354    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-oauth-config\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.222435 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.222386    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-serving-cert\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.223119 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.223087    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-oauth-serving-cert\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.223259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.223140    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-console-config\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.223259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.223160    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-service-ca\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.223367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.223271    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-trusted-ca-bundle\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.224951 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.224932    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-serving-cert\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.225025 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.224959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-oauth-config\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.230809 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.230780    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kf86s\" (UniqueName: \"kubernetes.io/projected/384e61b4-a5e3-43d6-8210-aa920b896545-kube-api-access-kf86s\") pod \"console-5c9cc5d77f-6dw7k\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") " pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.357567 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.357532    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:26.636598 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.636575    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c9cc5d77f-6dw7k"]
Sep 10 14:26:26.639045 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:26:26.639020    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod384e61b4_a5e3_43d6_8210_aa920b896545.slice/crio-23e3a566904ffca2c62017733fb57316aba5be6361dde50d62d7fad1f486f3d2 WatchSource:0}: Error finding container 23e3a566904ffca2c62017733fb57316aba5be6361dde50d62d7fad1f486f3d2: Status 404 returned error can't find the container with id 23e3a566904ffca2c62017733fb57316aba5be6361dde50d62d7fad1f486f3d2
Sep 10 14:26:26.732548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.732471    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:26:26.732834 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.732808    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="alertmanager" containerID="cri-o://23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a" gracePeriod=120
Sep 10 14:26:26.732904 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.732879    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy-metric" containerID="cri-o://1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b" gracePeriod=120
Sep 10 14:26:26.732949 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.732899    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy" containerID="cri-o://2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692" gracePeriod=120
Sep 10 14:26:26.732949 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.732899    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="config-reloader" containerID="cri-o://427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696" gracePeriod=120
Sep 10 14:26:26.733039 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.732879    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy-web" containerID="cri-o://b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379" gracePeriod=120
Sep 10 14:26:26.733039 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:26.733018    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="prom-label-proxy" containerID="cri-o://693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33" gracePeriod=120
Sep 10 14:26:27.114137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.114098    2578 generic.go:356] "Generic (PLEG): container finished" podID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerID="693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33" exitCode=0
Sep 10 14:26:27.114137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.114126    2578 generic.go:356] "Generic (PLEG): container finished" podID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerID="2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692" exitCode=0
Sep 10 14:26:27.114137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.114134    2578 generic.go:356] "Generic (PLEG): container finished" podID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerID="427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696" exitCode=0
Sep 10 14:26:27.114631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.114145    2578 generic.go:356] "Generic (PLEG): container finished" podID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerID="23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a" exitCode=0
Sep 10 14:26:27.114631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.114163    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerDied","Data":"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33"}
Sep 10 14:26:27.114631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.114241    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerDied","Data":"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692"}
Sep 10 14:26:27.114631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.114289    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerDied","Data":"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696"}
Sep 10 14:26:27.114631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.114313    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerDied","Data":"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a"}
Sep 10 14:26:27.115817 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.115781    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9cc5d77f-6dw7k" event={"ID":"384e61b4-a5e3-43d6-8210-aa920b896545","Type":"ContainerStarted","Data":"10d9e03357e12dc6ed541675de99d43641180d9db68eece9d95d581fd1a7810a"}
Sep 10 14:26:27.115951 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.115828    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9cc5d77f-6dw7k" event={"ID":"384e61b4-a5e3-43d6-8210-aa920b896545","Type":"ContainerStarted","Data":"23e3a566904ffca2c62017733fb57316aba5be6361dde50d62d7fad1f486f3d2"}
Sep 10 14:26:27.994548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:27.994519    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.023696 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.023614    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c9cc5d77f-6dw7k" podStartSLOduration=2.023590514 podStartE2EDuration="2.023590514s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:27.145230014 +0000 UTC m=+392.912952795" watchObservedRunningTime="2026-09-10 14:26:28.023590514 +0000 UTC m=+393.791313296"
Sep 10 14:26:28.039302 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039270    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039479 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039336    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtccj\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-kube-api-access-gtccj\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039479 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039358    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-cluster-tls-config\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039479 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039444    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-web\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039633 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039479    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-volume\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039633 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039502    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039633 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039540    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-trusted-ca-bundle\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039633 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039594    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039651    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-out\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039680    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-main-db\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039704    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-metrics-client-ca\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039775    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-web-config\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.039871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.039810    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-tls-assets\") pod \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\" (UID: \"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60\") "
Sep 10 14:26:28.040915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.040169    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:26:28.040915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.040280    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:26:28.040915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.040627    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:26:28.042769 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.042722    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:26:28.043140 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.043115    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-out" (OuterVolumeSpecName: "config-out") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:26:28.043539 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.043515    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-kube-api-access-gtccj" (OuterVolumeSpecName: "kube-api-access-gtccj") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "kube-api-access-gtccj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:26:28.043900 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.043873    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:26:28.044918 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.044849    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:26:28.045015 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.044956    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:26:28.045074 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.045031    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-volume" (OuterVolumeSpecName: "config-volume") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:26:28.045255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.045233    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:26:28.047621 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.047592    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:26:28.054721 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.054701    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-web-config" (OuterVolumeSpecName: "web-config") pod "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" (UID: "c925a7e5-4e7d-49c2-a89c-cbf2e973fb60"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:26:28.122242 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.122202    2578 generic.go:356] "Generic (PLEG): container finished" podID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerID="1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b" exitCode=0
Sep 10 14:26:28.122242 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.122233    2578 generic.go:356] "Generic (PLEG): container finished" podID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerID="b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379" exitCode=0
Sep 10 14:26:28.122787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.122291    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.122787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.122294    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerDied","Data":"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b"}
Sep 10 14:26:28.122787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.122399    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerDied","Data":"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379"}
Sep 10 14:26:28.122787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.122427    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"c925a7e5-4e7d-49c2-a89c-cbf2e973fb60","Type":"ContainerDied","Data":"0827923f74a1a179b01b2afeab160e0700ceac5f54027ba6584721b7c7bf54fc"}
Sep 10 14:26:28.122787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.122460    2578 scope.go:117] "RemoveContainer" containerID="693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33"
Sep 10 14:26:28.130210 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.130195    2578 scope.go:117] "RemoveContainer" containerID="1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b"
Sep 10 14:26:28.137568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.137549    2578 scope.go:117] "RemoveContainer" containerID="2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692"
Sep 10 14:26:28.140779 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140759    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.140879 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140782    2578 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-out\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.140879 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140816    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-main-db\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.140879 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140830    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-metrics-client-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.140879 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140867    2578 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-web-config\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.141047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140882    2578 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-tls-assets\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.141047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140896    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.141047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140910    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtccj\" (UniqueName: \"kubernetes.io/projected/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-kube-api-access-gtccj\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.141047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140924    2578 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-cluster-tls-config\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.141047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140938    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.141047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140950    2578 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.141047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140964    2578 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-secret-alertmanager-main-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.141047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.140977    2578 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:26:28.145069 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.145049    2578 scope.go:117] "RemoveContainer" containerID="b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379"
Sep 10 14:26:28.151958 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.151928    2578 scope.go:117] "RemoveContainer" containerID="427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696"
Sep 10 14:26:28.159028 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.159004    2578 scope.go:117] "RemoveContainer" containerID="23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a"
Sep 10 14:26:28.164664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.164641    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:26:28.166887 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.166864    2578 scope.go:117] "RemoveContainer" containerID="cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be"
Sep 10 14:26:28.174043 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.174019    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:26:28.174283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.174266    2578 scope.go:117] "RemoveContainer" containerID="693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33"
Sep 10 14:26:28.174551 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:26:28.174532    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33\": container with ID starting with 693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33 not found: ID does not exist" containerID="693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33"
Sep 10 14:26:28.174618 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.174559    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33"} err="failed to get container status \"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33\": rpc error: code = NotFound desc = could not find container \"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33\": container with ID starting with 693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33 not found: ID does not exist"
Sep 10 14:26:28.174618 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.174578    2578 scope.go:117] "RemoveContainer" containerID="1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b"
Sep 10 14:26:28.174854 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:26:28.174838    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b\": container with ID starting with 1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b not found: ID does not exist" containerID="1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b"
Sep 10 14:26:28.174892 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.174860    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b"} err="failed to get container status \"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b\": rpc error: code = NotFound desc = could not find container \"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b\": container with ID starting with 1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b not found: ID does not exist"
Sep 10 14:26:28.174892 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.174875    2578 scope.go:117] "RemoveContainer" containerID="2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692"
Sep 10 14:26:28.175119 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:26:28.175101    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692\": container with ID starting with 2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692 not found: ID does not exist" containerID="2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692"
Sep 10 14:26:28.175169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.175123    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692"} err="failed to get container status \"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692\": rpc error: code = NotFound desc = could not find container \"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692\": container with ID starting with 2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692 not found: ID does not exist"
Sep 10 14:26:28.175169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.175137    2578 scope.go:117] "RemoveContainer" containerID="b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379"
Sep 10 14:26:28.175380 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:26:28.175366    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379\": container with ID starting with b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379 not found: ID does not exist" containerID="b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379"
Sep 10 14:26:28.175416 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.175383    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379"} err="failed to get container status \"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379\": rpc error: code = NotFound desc = could not find container \"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379\": container with ID starting with b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379 not found: ID does not exist"
Sep 10 14:26:28.175416 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.175395    2578 scope.go:117] "RemoveContainer" containerID="427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696"
Sep 10 14:26:28.175602 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:26:28.175586    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696\": container with ID starting with 427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696 not found: ID does not exist" containerID="427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696"
Sep 10 14:26:28.175640 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.175606    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696"} err="failed to get container status \"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696\": rpc error: code = NotFound desc = could not find container \"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696\": container with ID starting with 427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696 not found: ID does not exist"
Sep 10 14:26:28.175640 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.175618    2578 scope.go:117] "RemoveContainer" containerID="23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a"
Sep 10 14:26:28.175903 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:26:28.175887    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a\": container with ID starting with 23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a not found: ID does not exist" containerID="23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a"
Sep 10 14:26:28.175953 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.175906    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a"} err="failed to get container status \"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a\": rpc error: code = NotFound desc = could not find container \"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a\": container with ID starting with 23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a not found: ID does not exist"
Sep 10 14:26:28.175953 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.175918    2578 scope.go:117] "RemoveContainer" containerID="cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be"
Sep 10 14:26:28.176143 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:26:28.176129    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be\": container with ID starting with cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be not found: ID does not exist" containerID="cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be"
Sep 10 14:26:28.176176 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.176150    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be"} err="failed to get container status \"cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be\": rpc error: code = NotFound desc = could not find container \"cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be\": container with ID starting with cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be not found: ID does not exist"
Sep 10 14:26:28.176176 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.176164    2578 scope.go:117] "RemoveContainer" containerID="693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33"
Sep 10 14:26:28.176397 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.176371    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33"} err="failed to get container status \"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33\": rpc error: code = NotFound desc = could not find container \"693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33\": container with ID starting with 693016bdafcd009c539a3587b4b880f5c06c8d3cdd7b2c12a868afb7e72e3b33 not found: ID does not exist"
Sep 10 14:26:28.176397 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.176395    2578 scope.go:117] "RemoveContainer" containerID="1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b"
Sep 10 14:26:28.176649 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.176631    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b"} err="failed to get container status \"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b\": rpc error: code = NotFound desc = could not find container \"1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b\": container with ID starting with 1747202e9e4244dd10f8c31b6567840d9572b21bfd878d643b33bd05a636f63b not found: ID does not exist"
Sep 10 14:26:28.176690 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.176650    2578 scope.go:117] "RemoveContainer" containerID="2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692"
Sep 10 14:26:28.176893 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.176876    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692"} err="failed to get container status \"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692\": rpc error: code = NotFound desc = could not find container \"2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692\": container with ID starting with 2ec90c258e41c7c52c255d5e0eadbf0010dd1c8200fbbfdcc907ff91017cc692 not found: ID does not exist"
Sep 10 14:26:28.176944 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.176893    2578 scope.go:117] "RemoveContainer" containerID="b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379"
Sep 10 14:26:28.177131 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.177114    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379"} err="failed to get container status \"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379\": rpc error: code = NotFound desc = could not find container \"b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379\": container with ID starting with b5d20a0cce0f340d1e5e5701eef43f2f3f4b4fb1a67c2e85fa0f560bd217a379 not found: ID does not exist"
Sep 10 14:26:28.177172 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.177133    2578 scope.go:117] "RemoveContainer" containerID="427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696"
Sep 10 14:26:28.177342 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.177327    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696"} err="failed to get container status \"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696\": rpc error: code = NotFound desc = could not find container \"427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696\": container with ID starting with 427bad679bc2efe1fbef37b9012e1fb4caec9a355934e298d9e252f76e756696 not found: ID does not exist"
Sep 10 14:26:28.177378 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.177343    2578 scope.go:117] "RemoveContainer" containerID="23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a"
Sep 10 14:26:28.177542 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.177527    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a"} err="failed to get container status \"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a\": rpc error: code = NotFound desc = could not find container \"23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a\": container with ID starting with 23cdbadd901470039c0e4253ea7612f98dc40ea8772521ae55d3f221e672a93a not found: ID does not exist"
Sep 10 14:26:28.177622 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.177543    2578 scope.go:117] "RemoveContainer" containerID="cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be"
Sep 10 14:26:28.177727 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.177708    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be"} err="failed to get container status \"cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be\": rpc error: code = NotFound desc = could not find container \"cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be\": container with ID starting with cda5184bd2bccb40d51cde30a5244f241341bfb9f4fa3e24417aa58dd1a9e3be not found: ID does not exist"
Sep 10 14:26:28.242864 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.242837    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:26:28.243072 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243060    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="config-reloader"
Sep 10 14:26:28.243111 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243074    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="config-reloader"
Sep 10 14:26:28.243111 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243083    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy"
Sep 10 14:26:28.243111 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243090    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy"
Sep 10 14:26:28.243111 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243101    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="init-config-reloader"
Sep 10 14:26:28.243111 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243107    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="init-config-reloader"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243116    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy-web"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243121    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy-web"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243126    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy-metric"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243131    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy-metric"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243143    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="prom-label-proxy"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243148    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="prom-label-proxy"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243153    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="alertmanager"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243158    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="alertmanager"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243197    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="config-reloader"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243204    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243210    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="alertmanager"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243215    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy-web"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243220    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="kube-rbac-proxy-metric"
Sep 10 14:26:28.243255 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.243225    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" containerName="prom-label-proxy"
Sep 10 14:26:28.248397 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.248375    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.250785 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.250756    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 10 14:26:28.250911 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.250892    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7xdxb\""
Sep 10 14:26:28.251176 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.251157    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 10 14:26:28.251302 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.251187    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 10 14:26:28.251302 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.251219    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 10 14:26:28.251302 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.251289    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 10 14:26:28.251483 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.251324    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 10 14:26:28.251483 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.251438    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 10 14:26:28.251483 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.251475    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 10 14:26:28.261819 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.261795    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 10 14:26:28.275820 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.275785    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:26:28.342557 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342524    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8gr7q\" (UniqueName: \"kubernetes.io/projected/958214c3-8d97-4848-a91f-a062d0bcbdbb-kube-api-access-8gr7q\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.342734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342564    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.342734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342585    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.342734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342616    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-web-config\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.342734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342636    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/958214c3-8d97-4848-a91f-a062d0bcbdbb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.342734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342651    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/958214c3-8d97-4848-a91f-a062d0bcbdbb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.342734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342667    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958214c3-8d97-4848-a91f-a062d0bcbdbb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.342734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342685    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.343055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342796    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/958214c3-8d97-4848-a91f-a062d0bcbdbb-config-out\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.343055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342836    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.343055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-config-volume\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.343055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342901    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/958214c3-8d97-4848-a91f-a062d0bcbdbb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.343055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.342923    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443350 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443311    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/958214c3-8d97-4848-a91f-a062d0bcbdbb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443350 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443355    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443388    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8gr7q\" (UniqueName: \"kubernetes.io/projected/958214c3-8d97-4848-a91f-a062d0bcbdbb-kube-api-access-8gr7q\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443426    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443449    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-web-config\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443470    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/958214c3-8d97-4848-a91f-a062d0bcbdbb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443535    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/958214c3-8d97-4848-a91f-a062d0bcbdbb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443587    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958214c3-8d97-4848-a91f-a062d0bcbdbb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443968 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443618    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443968 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443677    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/958214c3-8d97-4848-a91f-a062d0bcbdbb-config-out\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443968 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443714    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.443968 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.443781    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-config-volume\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.445154 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.445123    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/958214c3-8d97-4848-a91f-a062d0bcbdbb-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.445698 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.445668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/958214c3-8d97-4848-a91f-a062d0bcbdbb-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.445816 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.445668    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/958214c3-8d97-4848-a91f-a062d0bcbdbb-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.446942 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.446914    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.447557 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.447327    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/958214c3-8d97-4848-a91f-a062d0bcbdbb-tls-assets\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.447557 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.447504    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.447557 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.447518    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-web-config\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.447704 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.447575    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-config-volume\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.447704 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.447637    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.447704 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.447694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.448623 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.448599    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/958214c3-8d97-4848-a91f-a062d0bcbdbb-config-out\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.448706 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.448688    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/958214c3-8d97-4848-a91f-a062d0bcbdbb-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.451294 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.451275    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8gr7q\" (UniqueName: \"kubernetes.io/projected/958214c3-8d97-4848-a91f-a062d0bcbdbb-kube-api-access-8gr7q\") pod \"alertmanager-main-0\" (UID: \"958214c3-8d97-4848-a91f-a062d0bcbdbb\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.558181 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.558139    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 10 14:26:28.722552 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.722515    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c925a7e5-4e7d-49c2-a89c-cbf2e973fb60" path="/var/lib/kubelet/pods/c925a7e5-4e7d-49c2-a89c-cbf2e973fb60/volumes"
Sep 10 14:26:28.841954 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:28.841918    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 10 14:26:28.845893 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:26:28.845859    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod958214c3_8d97_4848_a91f_a062d0bcbdbb.slice/crio-eeeb1966c95e4f6745e5f1f57900fd4514d72647b394f6b650d813ddb5f9f679 WatchSource:0}: Error finding container eeeb1966c95e4f6745e5f1f57900fd4514d72647b394f6b650d813ddb5f9f679: Status 404 returned error can't find the container with id eeeb1966c95e4f6745e5f1f57900fd4514d72647b394f6b650d813ddb5f9f679
Sep 10 14:26:29.129315 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:29.129223    2578 generic.go:356] "Generic (PLEG): container finished" podID="958214c3-8d97-4848-a91f-a062d0bcbdbb" containerID="9883dbe4d6444d31ed1f27135c1bea7c6b1f587fddbef3995af350ec621293da" exitCode=0
Sep 10 14:26:29.129315 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:29.129282    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"958214c3-8d97-4848-a91f-a062d0bcbdbb","Type":"ContainerDied","Data":"9883dbe4d6444d31ed1f27135c1bea7c6b1f587fddbef3995af350ec621293da"}
Sep 10 14:26:29.129711 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:29.129352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"958214c3-8d97-4848-a91f-a062d0bcbdbb","Type":"ContainerStarted","Data":"eeeb1966c95e4f6745e5f1f57900fd4514d72647b394f6b650d813ddb5f9f679"}
Sep 10 14:26:30.137137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:30.137091    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"958214c3-8d97-4848-a91f-a062d0bcbdbb","Type":"ContainerStarted","Data":"9456290fe651dfad16e3efffe2160bb4e60000ddeaaa489e327244bdcc81180a"}
Sep 10 14:26:30.137635 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:30.137221    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"958214c3-8d97-4848-a91f-a062d0bcbdbb","Type":"ContainerStarted","Data":"7d5e68d6bc070c9bc37185fd9c35c601a0ec8e511f0dcd8dbf1b4dab19199f27"}
Sep 10 14:26:30.137635 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:30.137264    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"958214c3-8d97-4848-a91f-a062d0bcbdbb","Type":"ContainerStarted","Data":"cfc6e3d8597b57cf037d000aa1bc2d93fb3a0aea973af88e73fb95503d3b92ec"}
Sep 10 14:26:30.137635 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:30.137314    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"958214c3-8d97-4848-a91f-a062d0bcbdbb","Type":"ContainerStarted","Data":"e993711fd668d50112e310e5064fb8426914d31c7d52fbcc8015e131762ca3c7"}
Sep 10 14:26:30.137635 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:30.137355    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"958214c3-8d97-4848-a91f-a062d0bcbdbb","Type":"ContainerStarted","Data":"ba8ba72419a52e60b55b3b0884677f1c35ac6794f3b0e33b38786c0102417fa9"}
Sep 10 14:26:30.137635 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:30.137396    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"958214c3-8d97-4848-a91f-a062d0bcbdbb","Type":"ContainerStarted","Data":"ea3e793d572ec89f004f5df38117780c411a520a3e0824010a835c61b5518ebc"}
Sep 10 14:26:30.166184 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:30.166131    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.148165321 podStartE2EDuration="2.166111748s" totalImagesPullingTime="0s" totalInitContainerRuntime="17.946427ms" isStatefulPod=false podCreationTimestamp="2026-09-10 14:26:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:26:30.165632598 +0000 UTC m=+395.933355381" watchObservedRunningTime="2026-09-10 14:26:30.166111748 +0000 UTC m=+395.933834531"
Sep 10 14:26:33.766801 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:33.766734    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:34.279432 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:34.279400    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:34.377774 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:34.377722    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:36.357713 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:36.357674    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:36.358120 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:36.357737    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:36.362193 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:36.362173    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:37.175666 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:37.175635    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:26:37.338028 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:37.337962    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:37.437902 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:37.437825    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:37.499793 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:37.499734    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:38.259058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:38.259018    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:38.360418 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:38.360385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:38.461390 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:38.461341    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:58.128206 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:58.128165    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:26:58.189581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:26:58.189541    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:21.711947 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:27:21.711862    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:21.770700 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:27:21.770664    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:25.907674 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:27:25.907637    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:27:26.005737 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:27:26.005709    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:28:14.403588 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:28:14.403550    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:49.100627 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:29:49.100580    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:49.152965 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:29:49.152936    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:29:52.717536 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:29:52.717495    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:08.511429 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:08.511391    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:10.761053 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:10.761017    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:43.681264 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:43.681232    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:43.730931 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:43.730900    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:54.929765 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:54.929710    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c9cc5d77f-6dw7k"]
Sep 10 14:30:55.037940 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:55.037907    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:55.090110 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:55.090076    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:55.150899 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:55.150876    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:56.614706 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:56.614675    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:30:56.924657 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:30:56.924563    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:18.489197 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:18.489154    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:19.512000 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:19.511956    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:19.949082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:19.949038    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5c9cc5d77f-6dw7k" podUID="384e61b4-a5e3-43d6-8210-aa920b896545" containerName="console" containerID="cri-o://10d9e03357e12dc6ed541675de99d43641180d9db68eece9d95d581fd1a7810a" gracePeriod=15
Sep 10 14:31:20.134741 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.134707    2578 generic.go:356] "Generic (PLEG): container finished" podID="384e61b4-a5e3-43d6-8210-aa920b896545" containerID="10d9e03357e12dc6ed541675de99d43641180d9db68eece9d95d581fd1a7810a" exitCode=0
Sep 10 14:31:20.134911 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.134779    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9cc5d77f-6dw7k" event={"ID":"384e61b4-a5e3-43d6-8210-aa920b896545","Type":"ContainerDied","Data":"10d9e03357e12dc6ed541675de99d43641180d9db68eece9d95d581fd1a7810a"}
Sep 10 14:31:20.178555 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.178529    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:31:20.242205 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242132    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kf86s\" (UniqueName: \"kubernetes.io/projected/384e61b4-a5e3-43d6-8210-aa920b896545-kube-api-access-kf86s\") pod \"384e61b4-a5e3-43d6-8210-aa920b896545\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") "
Sep 10 14:31:20.242205 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242163    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-service-ca\") pod \"384e61b4-a5e3-43d6-8210-aa920b896545\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") "
Sep 10 14:31:20.242205 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242185    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-oauth-serving-cert\") pod \"384e61b4-a5e3-43d6-8210-aa920b896545\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") "
Sep 10 14:31:20.242427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242216    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-trusted-ca-bundle\") pod \"384e61b4-a5e3-43d6-8210-aa920b896545\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") "
Sep 10 14:31:20.242427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242303    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-oauth-config\") pod \"384e61b4-a5e3-43d6-8210-aa920b896545\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") "
Sep 10 14:31:20.242427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242354    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-console-config\") pod \"384e61b4-a5e3-43d6-8210-aa920b896545\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") "
Sep 10 14:31:20.242427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242380    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-serving-cert\") pod \"384e61b4-a5e3-43d6-8210-aa920b896545\" (UID: \"384e61b4-a5e3-43d6-8210-aa920b896545\") "
Sep 10 14:31:20.242596 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242529    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "384e61b4-a5e3-43d6-8210-aa920b896545" (UID: "384e61b4-a5e3-43d6-8210-aa920b896545"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:20.242650 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242607    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-service-ca" (OuterVolumeSpecName: "service-ca") pod "384e61b4-a5e3-43d6-8210-aa920b896545" (UID: "384e61b4-a5e3-43d6-8210-aa920b896545"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:20.242814 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242794    2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:31:20.242886 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242819    2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-oauth-serving-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:31:20.242886 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242824    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-console-config" (OuterVolumeSpecName: "console-config") pod "384e61b4-a5e3-43d6-8210-aa920b896545" (UID: "384e61b4-a5e3-43d6-8210-aa920b896545"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:20.242964 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.242951    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "384e61b4-a5e3-43d6-8210-aa920b896545" (UID: "384e61b4-a5e3-43d6-8210-aa920b896545"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:31:20.244399 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.244383    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/384e61b4-a5e3-43d6-8210-aa920b896545-kube-api-access-kf86s" (OuterVolumeSpecName: "kube-api-access-kf86s") pod "384e61b4-a5e3-43d6-8210-aa920b896545" (UID: "384e61b4-a5e3-43d6-8210-aa920b896545"). InnerVolumeSpecName "kube-api-access-kf86s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:31:20.244982 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.244962    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "384e61b4-a5e3-43d6-8210-aa920b896545" (UID: "384e61b4-a5e3-43d6-8210-aa920b896545"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:20.245050 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.244987    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "384e61b4-a5e3-43d6-8210-aa920b896545" (UID: "384e61b4-a5e3-43d6-8210-aa920b896545"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:31:20.343506 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.343471    2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-oauth-config\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:31:20.343506 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.343502    2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-console-config\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:31:20.343506 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.343511    2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/384e61b4-a5e3-43d6-8210-aa920b896545-console-serving-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:31:20.343694 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.343520    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kf86s\" (UniqueName: \"kubernetes.io/projected/384e61b4-a5e3-43d6-8210-aa920b896545-kube-api-access-kf86s\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:31:20.343694 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.343529    2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/384e61b4-a5e3-43d6-8210-aa920b896545-trusted-ca-bundle\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:31:20.711077 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.711041    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"]
Sep 10 14:31:20.711509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.711290    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="384e61b4-a5e3-43d6-8210-aa920b896545" containerName="console"
Sep 10 14:31:20.711509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.711300    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="384e61b4-a5e3-43d6-8210-aa920b896545" containerName="console"
Sep 10 14:31:20.711509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.711362    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="384e61b4-a5e3-43d6-8210-aa920b896545" containerName="console"
Sep 10 14:31:20.714365 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.714345    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"
Sep 10 14:31:20.716763 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.716730    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:31:20.717103 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.717082    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-4w9r4\""
Sep 10 14:31:20.717222 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.717208    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
Sep 10 14:31:20.734302 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.734279    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"]
Sep 10 14:31:20.746259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.746233    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d130e173-11db-45ab-b8ea-f00f15a19378-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vv77x\" (UID: \"d130e173-11db-45ab-b8ea-f00f15a19378\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"
Sep 10 14:31:20.746377 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.746359    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbggn\" (UniqueName: \"kubernetes.io/projected/d130e173-11db-45ab-b8ea-f00f15a19378-kube-api-access-cbggn\") pod \"cert-manager-operator-controller-manager-55c444bf86-vv77x\" (UID: \"d130e173-11db-45ab-b8ea-f00f15a19378\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"
Sep 10 14:31:20.847243 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.847207    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbggn\" (UniqueName: \"kubernetes.io/projected/d130e173-11db-45ab-b8ea-f00f15a19378-kube-api-access-cbggn\") pod \"cert-manager-operator-controller-manager-55c444bf86-vv77x\" (UID: \"d130e173-11db-45ab-b8ea-f00f15a19378\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"
Sep 10 14:31:20.847425 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.847258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d130e173-11db-45ab-b8ea-f00f15a19378-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vv77x\" (UID: \"d130e173-11db-45ab-b8ea-f00f15a19378\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"
Sep 10 14:31:20.847634 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.847614    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d130e173-11db-45ab-b8ea-f00f15a19378-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-vv77x\" (UID: \"d130e173-11db-45ab-b8ea-f00f15a19378\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"
Sep 10 14:31:20.858411 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:20.858382    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbggn\" (UniqueName: \"kubernetes.io/projected/d130e173-11db-45ab-b8ea-f00f15a19378-kube-api-access-cbggn\") pod \"cert-manager-operator-controller-manager-55c444bf86-vv77x\" (UID: \"d130e173-11db-45ab-b8ea-f00f15a19378\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"
Sep 10 14:31:21.023462 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:21.023376    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"
Sep 10 14:31:21.139188 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:21.139146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c9cc5d77f-6dw7k" event={"ID":"384e61b4-a5e3-43d6-8210-aa920b896545","Type":"ContainerDied","Data":"23e3a566904ffca2c62017733fb57316aba5be6361dde50d62d7fad1f486f3d2"}
Sep 10 14:31:21.139376 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:21.139197    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c9cc5d77f-6dw7k"
Sep 10 14:31:21.139376 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:21.139234    2578 scope.go:117] "RemoveContainer" containerID="10d9e03357e12dc6ed541675de99d43641180d9db68eece9d95d581fd1a7810a"
Sep 10 14:31:21.159502 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:21.159480    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c9cc5d77f-6dw7k"]
Sep 10 14:31:21.166187 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:21.166164    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c9cc5d77f-6dw7k"]
Sep 10 14:31:21.302415 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:21.302336    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x"]
Sep 10 14:31:21.309395 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:31:21.309368    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd130e173_11db_45ab_b8ea_f00f15a19378.slice/crio-66e96143e6b257dad0ff7a90c55d3806c93fbb4cde8d99e491b7649913458a5f WatchSource:0}: Error finding container 66e96143e6b257dad0ff7a90c55d3806c93fbb4cde8d99e491b7649913458a5f: Status 404 returned error can't find the container with id 66e96143e6b257dad0ff7a90c55d3806c93fbb4cde8d99e491b7649913458a5f
Sep 10 14:31:21.311942 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:21.311926    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:31:22.143950 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:22.143915    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x" event={"ID":"d130e173-11db-45ab-b8ea-f00f15a19378","Type":"ContainerStarted","Data":"66e96143e6b257dad0ff7a90c55d3806c93fbb4cde8d99e491b7649913458a5f"}
Sep 10 14:31:22.310234 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:22.310187    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:22.402558 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:22.402461    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:22.725250 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:22.725164    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="384e61b4-a5e3-43d6-8210-aa920b896545" path="/var/lib/kubelet/pods/384e61b4-a5e3-43d6-8210-aa920b896545/volumes"
Sep 10 14:31:25.089499 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:25.089467    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:25.160060 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:25.160021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x" event={"ID":"d130e173-11db-45ab-b8ea-f00f15a19378","Type":"ContainerStarted","Data":"3997e5fb8e4df770b61ca0177b651f8bd47935711b640310560a8d6801f6db71"}
Sep 10 14:31:25.186761 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:25.186691    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-vv77x" podStartSLOduration=2.129005063 podStartE2EDuration="5.186673806s" totalImagesPullingTime="3.057668743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:31:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:31:25.18599011 +0000 UTC m=+690.953712884" watchObservedRunningTime="2026-09-10 14:31:25.186673806 +0000 UTC m=+690.954396584"
Sep 10 14:31:25.203765 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:25.203714    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:25.653683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:25.653652    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:25.773366 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:25.773328    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:27.341729 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:27.341693    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:27.629027 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:27.628944    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:28.101116 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:28.101083    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:28.166636 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:28.166598    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:34.951318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:34.951274    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:35.049064 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:35.049030    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:35.627696 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:35.627668    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:36.628858 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.628823    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"]
Sep 10 14:31:36.634621 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.634605    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"
Sep 10 14:31:36.636800 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.636780    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 10 14:31:36.636905 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.636799    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2h7nb\""
Sep 10 14:31:36.637137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.637125    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 10 14:31:36.643978 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.643959    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"]
Sep 10 14:31:36.780315 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.780265    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sfq8s\" (UniqueName: \"kubernetes.io/projected/d1bc2c98-b7e8-4b15-a39c-d532dcb85455-kube-api-access-sfq8s\") pod \"openshift-lws-operator-fd8ccff4c-tbcnp\" (UID: \"d1bc2c98-b7e8-4b15-a39c-d532dcb85455\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"
Sep 10 14:31:36.780498 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.780350    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1bc2c98-b7e8-4b15-a39c-d532dcb85455-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tbcnp\" (UID: \"d1bc2c98-b7e8-4b15-a39c-d532dcb85455\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"
Sep 10 14:31:36.881100 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.881009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1bc2c98-b7e8-4b15-a39c-d532dcb85455-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tbcnp\" (UID: \"d1bc2c98-b7e8-4b15-a39c-d532dcb85455\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"
Sep 10 14:31:36.881100 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.881078    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sfq8s\" (UniqueName: \"kubernetes.io/projected/d1bc2c98-b7e8-4b15-a39c-d532dcb85455-kube-api-access-sfq8s\") pod \"openshift-lws-operator-fd8ccff4c-tbcnp\" (UID: \"d1bc2c98-b7e8-4b15-a39c-d532dcb85455\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"
Sep 10 14:31:36.881411 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.881389    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d1bc2c98-b7e8-4b15-a39c-d532dcb85455-tmp\") pod \"openshift-lws-operator-fd8ccff4c-tbcnp\" (UID: \"d1bc2c98-b7e8-4b15-a39c-d532dcb85455\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"
Sep 10 14:31:36.888967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.888941    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sfq8s\" (UniqueName: \"kubernetes.io/projected/d1bc2c98-b7e8-4b15-a39c-d532dcb85455-kube-api-access-sfq8s\") pod \"openshift-lws-operator-fd8ccff4c-tbcnp\" (UID: \"d1bc2c98-b7e8-4b15-a39c-d532dcb85455\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"
Sep 10 14:31:36.942891 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:36.942854    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"
Sep 10 14:31:37.216429 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:37.216349    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp"]
Sep 10 14:31:37.219817 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:31:37.219793    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1bc2c98_b7e8_4b15_a39c_d532dcb85455.slice/crio-edfb12d8ae3c3434200130034b660be9aa5a5748cd629378016f2e541280036f WatchSource:0}: Error finding container edfb12d8ae3c3434200130034b660be9aa5a5748cd629378016f2e541280036f: Status 404 returned error can't find the container with id edfb12d8ae3c3434200130034b660be9aa5a5748cd629378016f2e541280036f
Sep 10 14:31:38.215070 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:38.215023    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp" event={"ID":"d1bc2c98-b7e8-4b15-a39c-d532dcb85455","Type":"ContainerStarted","Data":"edfb12d8ae3c3434200130034b660be9aa5a5748cd629378016f2e541280036f"}
Sep 10 14:31:40.225714 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:40.225673    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp" event={"ID":"d1bc2c98-b7e8-4b15-a39c-d532dcb85455","Type":"ContainerStarted","Data":"aba195dad1ed525d612530c911f495c2e32fabc64b00af6bcaf56bad6e427fa2"}
Sep 10 14:31:40.246022 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:40.245958    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-tbcnp" podStartSLOduration=1.711119271 podStartE2EDuration="4.245934666s" totalImagesPullingTime="2.534815395s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:31:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:31:40.24535422 +0000 UTC m=+706.013077012" watchObservedRunningTime="2026-09-10 14:31:40.245934666 +0000 UTC m=+706.013657448"
Sep 10 14:31:40.989635 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:40.989602    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:41.089455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:41.089422    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:43.182235 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:43.182197    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:45.110990 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:45.110943    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:45.212110 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:45.212073    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:54.307817 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:54.307781    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:54.408474 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:54.408443    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:54.852161 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:54.852127    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:55.161525 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.161488    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:55.317445 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.317410    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:55.325952 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.325922    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"]
Sep 10 14:31:55.330304 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.330282    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.332250 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.332224    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 10 14:31:55.332470 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.332451    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 10 14:31:55.332568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.332451    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-6jvst\""
Sep 10 14:31:55.332568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.332538    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 10 14:31:55.334674 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.334658    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 10 14:31:55.349212 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.349186    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"]
Sep 10 14:31:55.435085 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.435006    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb64b04-52f6-4612-b545-ab66d06106f2-webhook-cert\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.435085 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.435075    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb64b04-52f6-4612-b545-ab66d06106f2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.435290 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.435103    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb64b04-52f6-4612-b545-ab66d06106f2-tmp\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.435290 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.435122    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb64b04-52f6-4612-b545-ab66d06106f2-manifests\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.435290 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.435137    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67l7z\" (UniqueName: \"kubernetes.io/projected/7cb64b04-52f6-4612-b545-ab66d06106f2-kube-api-access-67l7z\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.500216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.500186    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:55.536527 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.536496    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb64b04-52f6-4612-b545-ab66d06106f2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.536685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.536535    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb64b04-52f6-4612-b545-ab66d06106f2-tmp\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.536685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.536567    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb64b04-52f6-4612-b545-ab66d06106f2-manifests\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.536685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.536590    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67l7z\" (UniqueName: \"kubernetes.io/projected/7cb64b04-52f6-4612-b545-ab66d06106f2-kube-api-access-67l7z\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.536685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.536637    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb64b04-52f6-4612-b545-ab66d06106f2-webhook-cert\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.536994 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.536971    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/7cb64b04-52f6-4612-b545-ab66d06106f2-manifests\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.539211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.539188    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7cb64b04-52f6-4612-b545-ab66d06106f2-tmp\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.539352 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.539336    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7cb64b04-52f6-4612-b545-ab66d06106f2-webhook-cert\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.539396 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.539357    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/7cb64b04-52f6-4612-b545-ab66d06106f2-apiservice-cert\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.547587 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.547567    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67l7z\" (UniqueName: \"kubernetes.io/projected/7cb64b04-52f6-4612-b545-ab66d06106f2-kube-api-access-67l7z\") pod \"opendatahub-operator-controller-manager-755bd7f47-vjwjf\" (UID: \"7cb64b04-52f6-4612-b545-ab66d06106f2\") " pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.595208 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.595179    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:55.640216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.640180    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:31:55.701941 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.701904    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:55.799592 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.799556    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:55.928281 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.928248    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"]
Sep 10 14:31:55.932548 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:31:55.932520    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cb64b04_52f6_4612_b545_ab66d06106f2.slice/crio-e9943ec473c0d29407f3b336502467c70362f67a0c483c39fcb760f121a22202 WatchSource:0}: Error finding container e9943ec473c0d29407f3b336502467c70362f67a0c483c39fcb760f121a22202: Status 404 returned error can't find the container with id e9943ec473c0d29407f3b336502467c70362f67a0c483c39fcb760f121a22202
Sep 10 14:31:55.951968 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:55.951939    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:56.050040 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:56.050012    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:31:56.290929 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:56.290848    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf" event={"ID":"7cb64b04-52f6-4612-b545-ab66d06106f2","Type":"ContainerStarted","Data":"e9943ec473c0d29407f3b336502467c70362f67a0c483c39fcb760f121a22202"}
Sep 10 14:31:59.306783 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:31:59.306731    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf" event={"ID":"7cb64b04-52f6-4612-b545-ab66d06106f2","Type":"ContainerStarted","Data":"8d6fce589a699b6f9e81077d46835b31c12641ed3ee8cb86c8db41acd54e2898"}
Sep 10 14:32:00.311929 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:00.311885    2578 generic.go:356] "Generic (PLEG): container finished" podID="7cb64b04-52f6-4612-b545-ab66d06106f2" containerID="8d6fce589a699b6f9e81077d46835b31c12641ed3ee8cb86c8db41acd54e2898" exitCode=0
Sep 10 14:32:00.312392 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:00.311970    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf" event={"ID":"7cb64b04-52f6-4612-b545-ab66d06106f2","Type":"ContainerDied","Data":"8d6fce589a699b6f9e81077d46835b31c12641ed3ee8cb86c8db41acd54e2898"}
Sep 10 14:32:01.318055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:01.318024    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf" event={"ID":"7cb64b04-52f6-4612-b545-ab66d06106f2","Type":"ContainerStarted","Data":"cd9441b4172d4936047c912e920b72ce9878296188c7daade4480d8c3d5024ee"}
Sep 10 14:32:01.318455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:01.318135    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:32:01.338609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:01.338548    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf" podStartSLOduration=2.050875749 podStartE2EDuration="6.338528787s" totalImagesPullingTime="3.247341638s" totalInitContainerRuntime="1.0403114s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:31:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:01.337873497 +0000 UTC m=+727.105596291" watchObservedRunningTime="2026-09-10 14:32:01.338528787 +0000 UTC m=+727.106251568"
Sep 10 14:32:09.845373 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:09.845334    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:09.947004 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:09.946967    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:12.325583 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:12.325548    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-755bd7f47-vjwjf"
Sep 10 14:32:14.974667 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:14.974634    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:15.092978 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:15.092944    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:15.977649 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:15.977617    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:16.014104 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.014069    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"]
Sep 10 14:32:16.026233 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.026210    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"]
Sep 10 14:32:16.026389 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.026318    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.028548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.028524    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 10 14:32:16.028680 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.028549    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-vgprh\""
Sep 10 14:32:16.028933 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.028915    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 10 14:32:16.029161 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.029144    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 10 14:32:16.029292 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.029267    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 10 14:32:16.092916 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.092869    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-tmp\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.093080 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.092933    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-46lng\" (UniqueName: \"kubernetes.io/projected/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-kube-api-access-46lng\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.093080 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.093005    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-tls-certs\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.193575 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.193540    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-tls-certs\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.193718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.193586    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-tmp\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.193718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.193632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-46lng\" (UniqueName: \"kubernetes.io/projected/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-kube-api-access-46lng\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.196107 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.196083    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-tmp\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.196245 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.196226    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-tls-certs\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.201292 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.201270    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-46lng\" (UniqueName: \"kubernetes.io/projected/b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f-kube-api-access-46lng\") pod \"kube-auth-proxy-6f69648866-xgxnx\" (UID: \"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f\") " pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.220531 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.220504    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:16.336399 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.336364    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"
Sep 10 14:32:16.611208 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:16.611135    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6f69648866-xgxnx"]
Sep 10 14:32:16.614169 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:32:16.614142    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c5e213_92b0_4ef3_a8b7_15fa0c5cca6f.slice/crio-544bf537a8d2c04f12e81683858df24c9ece971d49559422345509bb103a34d5 WatchSource:0}: Error finding container 544bf537a8d2c04f12e81683858df24c9ece971d49559422345509bb103a34d5: Status 404 returned error can't find the container with id 544bf537a8d2c04f12e81683858df24c9ece971d49559422345509bb103a34d5
Sep 10 14:32:17.385615 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.385563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx" event={"ID":"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f","Type":"ContainerStarted","Data":"544bf537a8d2c04f12e81683858df24c9ece971d49559422345509bb103a34d5"}
Sep 10 14:32:17.475829 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.475787    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.479053 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.479026    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"]
Sep 10 14:32:17.481859 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.481840    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.484737 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.484709    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 10 14:32:17.484922 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.484902    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 10 14:32:17.485004 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.484987    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 10 14:32:17.485131 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.485114    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 10 14:32:17.485300 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.485256    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-879xk\""
Sep 10 14:32:17.496947 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.496920    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"]
Sep 10 14:32:17.504529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.504204    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.504529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.504259    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k7xn\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-kube-api-access-5k7xn\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.504529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.504290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.504529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.504321    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.504529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.504343    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.504529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.504377    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.504529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.504409    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86389b36-a953-43dd-add8-dbe1ba7ce747-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.572413 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.572354    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:17.605533 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.605498    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.605858 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.605559    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k7xn\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-kube-api-access-5k7xn\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.605858 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.605591    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.605858 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.605617    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.605858 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.605632    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.605858 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.605659    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.605858 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.605696    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86389b36-a953-43dd-add8-dbe1ba7ce747-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.607604 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.607533    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.610111 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.610082    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.611085 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.611032    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.611411 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.611387    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86389b36-a953-43dd-add8-dbe1ba7ce747-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.612343 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.612319    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.615596 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.615573    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k7xn\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-kube-api-access-5k7xn\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.616484 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.616462    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-49x7k\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:17.801958 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:17.801874    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:18.094529 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:18.094461    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"]
Sep 10 14:32:18.593014 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:18.592964    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:19.411840 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:32:19.411806    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86389b36_a953_43dd_add8_dbe1ba7ce747.slice/crio-87bc25ca42d33f57b64408651dbf742a627888fbcac8e79a72a5fac3a32d4712 WatchSource:0}: Error finding container 87bc25ca42d33f57b64408651dbf742a627888fbcac8e79a72a5fac3a32d4712: Status 404 returned error can't find the container with id 87bc25ca42d33f57b64408651dbf742a627888fbcac8e79a72a5fac3a32d4712
Sep 10 14:32:20.402470 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:20.402294    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx" event={"ID":"b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f","Type":"ContainerStarted","Data":"0221c6c9d02bc5bad954de2fdf71ed7f6c95e6b09b57499efcda6d3bad519d2b"}
Sep 10 14:32:20.404047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:20.403993    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k" event={"ID":"86389b36-a953-43dd-add8-dbe1ba7ce747","Type":"ContainerStarted","Data":"87bc25ca42d33f57b64408651dbf742a627888fbcac8e79a72a5fac3a32d4712"}
Sep 10 14:32:20.418770 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:20.418605    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6f69648866-xgxnx" podStartSLOduration=2.569482904 podStartE2EDuration="5.41858861s" totalImagesPullingTime="2.849105706s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:20.417462284 +0000 UTC m=+746.185185085" watchObservedRunningTime="2026-09-10 14:32:20.41858861 +0000 UTC m=+746.186311390"
Sep 10 14:32:22.047146 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:22.047083    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:22.047443 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:22.047166    2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892152Ki","pods":"250"}
Sep 10 14:32:22.415055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:22.415017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k" event={"ID":"86389b36-a953-43dd-add8-dbe1ba7ce747","Type":"ContainerStarted","Data":"df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159"}
Sep 10 14:32:22.415259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:22.415174    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:22.435766 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:22.435687    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k" podStartSLOduration=2.802630297 podStartE2EDuration="5.43567202s" totalImagesPullingTime="2.633041723s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:22.434264502 +0000 UTC m=+748.201987288" watchObservedRunningTime="2026-09-10 14:32:22.43567202 +0000 UTC m=+748.203394800"
Sep 10 14:32:22.791001 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:22.790902    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:22.863164 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:22.863126    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:23.084717 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:23.084683    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:23.177819 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:23.177780    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:23.285040 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:23.285006    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:23.421413 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:23.421335    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:26.013272 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:26.013230    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:26.112421 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:26.112393    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:26.212252 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:26.212215    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:26.312544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:26.312448    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:26.495310 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:26.495271    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:26.714031 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:26.713989    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:26.775581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:26.775545    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:27.463489 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:27.463454    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:35.838453 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.838423    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:35.931058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.931023    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"]
Sep 10 14:32:35.934583 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.934564    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:35.937628 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.937604    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jrmzj\""
Sep 10 14:32:35.939915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.939893    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 10 14:32:35.958716 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.958686    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"]
Sep 10 14:32:35.961825 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.961808    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:35.964774 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.964731    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 10 14:32:35.965801 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.965782    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-25nd4\""
Sep 10 14:32:35.967789 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.967740    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:35.969954 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.969930    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"]
Sep 10 14:32:35.999248 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:35.999205    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"]
Sep 10 14:32:36.062689 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.062648    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7v4mq\" (UniqueName: \"kubernetes.io/projected/a9446da5-a76a-4045-b2f7-b6461c1fb59d-kube-api-access-7v4mq\") pod \"kserve-controller-manager-d6f4b65b8-mnjns\" (UID: \"a9446da5-a76a-4045-b2f7-b6461c1fb59d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:36.062884 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.062701    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9446da5-a76a-4045-b2f7-b6461c1fb59d-cert\") pod \"kserve-controller-manager-d6f4b65b8-mnjns\" (UID: \"a9446da5-a76a-4045-b2f7-b6461c1fb59d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:36.062884 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.062727    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d2fr4\" (UID: \"37d62584-d7e1-4aed-9fcd-37ffbeaa687a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:36.062884 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.062814    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q429p\" (UniqueName: \"kubernetes.io/projected/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-kube-api-access-q429p\") pod \"llmisvc-controller-manager-777c6bd7d5-d2fr4\" (UID: \"37d62584-d7e1-4aed-9fcd-37ffbeaa687a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:36.163244 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.163140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d2fr4\" (UID: \"37d62584-d7e1-4aed-9fcd-37ffbeaa687a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:36.163244 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.163198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q429p\" (UniqueName: \"kubernetes.io/projected/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-kube-api-access-q429p\") pod \"llmisvc-controller-manager-777c6bd7d5-d2fr4\" (UID: \"37d62584-d7e1-4aed-9fcd-37ffbeaa687a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:36.163480 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.163272    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7v4mq\" (UniqueName: \"kubernetes.io/projected/a9446da5-a76a-4045-b2f7-b6461c1fb59d-kube-api-access-7v4mq\") pod \"kserve-controller-manager-d6f4b65b8-mnjns\" (UID: \"a9446da5-a76a-4045-b2f7-b6461c1fb59d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:36.163480 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:32:36.163279    2578 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 10 14:32:36.163480 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.163322    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9446da5-a76a-4045-b2f7-b6461c1fb59d-cert\") pod \"kserve-controller-manager-d6f4b65b8-mnjns\" (UID: \"a9446da5-a76a-4045-b2f7-b6461c1fb59d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:36.163480 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:32:36.163350    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-cert podName:37d62584-d7e1-4aed-9fcd-37ffbeaa687a nodeName:}" failed. No retries permitted until 2026-09-10 14:32:36.663330253 +0000 UTC m=+762.431053012 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-cert") pod "llmisvc-controller-manager-777c6bd7d5-d2fr4" (UID: "37d62584-d7e1-4aed-9fcd-37ffbeaa687a") : secret "llmisvc-webhook-server-cert" not found
Sep 10 14:32:36.165970 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.165938    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a9446da5-a76a-4045-b2f7-b6461c1fb59d-cert\") pod \"kserve-controller-manager-d6f4b65b8-mnjns\" (UID: \"a9446da5-a76a-4045-b2f7-b6461c1fb59d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:36.180915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.180890    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q429p\" (UniqueName: \"kubernetes.io/projected/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-kube-api-access-q429p\") pod \"llmisvc-controller-manager-777c6bd7d5-d2fr4\" (UID: \"37d62584-d7e1-4aed-9fcd-37ffbeaa687a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:36.181025 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.180985    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7v4mq\" (UniqueName: \"kubernetes.io/projected/a9446da5-a76a-4045-b2f7-b6461c1fb59d-kube-api-access-7v4mq\") pod \"kserve-controller-manager-d6f4b65b8-mnjns\" (UID: \"a9446da5-a76a-4045-b2f7-b6461c1fb59d\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:36.185782 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.185737    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:36.245372 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.245346    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:36.284817 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.284772    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:36.465378 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.465286    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:32:36.515900 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.515870    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"]
Sep 10 14:32:36.518738 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:32:36.518709    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9446da5_a76a_4045_b2f7_b6461c1fb59d.slice/crio-ce7b1d40e00a13fac13c1fd6e52d4d0b6f49dcb352b657f08ef5204f43793cf1 WatchSource:0}: Error finding container ce7b1d40e00a13fac13c1fd6e52d4d0b6f49dcb352b657f08ef5204f43793cf1: Status 404 returned error can't find the container with id ce7b1d40e00a13fac13c1fd6e52d4d0b6f49dcb352b657f08ef5204f43793cf1
Sep 10 14:32:36.667675 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.667638    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d2fr4\" (UID: \"37d62584-d7e1-4aed-9fcd-37ffbeaa687a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:36.670252 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.670232    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/37d62584-d7e1-4aed-9fcd-37ffbeaa687a-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-d2fr4\" (UID: \"37d62584-d7e1-4aed-9fcd-37ffbeaa687a\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:36.873446 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:36.873411    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:37.148842 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:37.148814    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"]
Sep 10 14:32:37.150899 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:32:37.150867    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod37d62584_d7e1_4aed_9fcd_37ffbeaa687a.slice/crio-74db54a85f781cd2bc2cf13ed08a5f337d6a404bb65ae79417631736756874ab WatchSource:0}: Error finding container 74db54a85f781cd2bc2cf13ed08a5f337d6a404bb65ae79417631736756874ab: Status 404 returned error can't find the container with id 74db54a85f781cd2bc2cf13ed08a5f337d6a404bb65ae79417631736756874ab
Sep 10 14:32:37.489648 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:37.489564    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns" event={"ID":"a9446da5-a76a-4045-b2f7-b6461c1fb59d","Type":"ContainerStarted","Data":"ce7b1d40e00a13fac13c1fd6e52d4d0b6f49dcb352b657f08ef5204f43793cf1"}
Sep 10 14:32:37.490789 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:37.490765    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4" event={"ID":"37d62584-d7e1-4aed-9fcd-37ffbeaa687a","Type":"ContainerStarted","Data":"74db54a85f781cd2bc2cf13ed08a5f337d6a404bb65ae79417631736756874ab"}
Sep 10 14:32:38.665261 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:38.665175    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"]
Sep 10 14:32:38.665730 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:38.665438    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k" podUID="86389b36-a953-43dd-add8-dbe1ba7ce747" containerName="discovery" containerID="cri-o://df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159" gracePeriod=30
Sep 10 14:32:39.317925 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.317899    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:39.393559 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.393520    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-ca-configmap\") pod \"86389b36-a953-43dd-add8-dbe1ba7ce747\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") "
Sep 10 14:32:39.393720 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.393594    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-dns-cert\") pod \"86389b36-a953-43dd-add8-dbe1ba7ce747\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") "
Sep 10 14:32:39.393720 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.393640    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86389b36-a953-43dd-add8-dbe1ba7ce747-local-certs\") pod \"86389b36-a953-43dd-add8-dbe1ba7ce747\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") "
Sep 10 14:32:39.393720 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.393666    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5k7xn\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-kube-api-access-5k7xn\") pod \"86389b36-a953-43dd-add8-dbe1ba7ce747\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") "
Sep 10 14:32:39.393720 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.393695    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-kubeconfig\") pod \"86389b36-a953-43dd-add8-dbe1ba7ce747\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") "
Sep 10 14:32:39.394004 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.393730    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-token\") pod \"86389b36-a953-43dd-add8-dbe1ba7ce747\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") "
Sep 10 14:32:39.394004 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.393812    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-cacerts\") pod \"86389b36-a953-43dd-add8-dbe1ba7ce747\" (UID: \"86389b36-a953-43dd-add8-dbe1ba7ce747\") "
Sep 10 14:32:39.394270 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.394234    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "86389b36-a953-43dd-add8-dbe1ba7ce747" (UID: "86389b36-a953-43dd-add8-dbe1ba7ce747"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:32:39.396941 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.396909    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-kube-api-access-5k7xn" (OuterVolumeSpecName: "kube-api-access-5k7xn") pod "86389b36-a953-43dd-add8-dbe1ba7ce747" (UID: "86389b36-a953-43dd-add8-dbe1ba7ce747"). InnerVolumeSpecName "kube-api-access-5k7xn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:39.397674 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.397625    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-cacerts" (OuterVolumeSpecName: "cacerts") pod "86389b36-a953-43dd-add8-dbe1ba7ce747" (UID: "86389b36-a953-43dd-add8-dbe1ba7ce747"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:32:39.397793 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.397707    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "86389b36-a953-43dd-add8-dbe1ba7ce747" (UID: "86389b36-a953-43dd-add8-dbe1ba7ce747"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:32:39.397793 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.397708    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86389b36-a953-43dd-add8-dbe1ba7ce747-local-certs" (OuterVolumeSpecName: "local-certs") pod "86389b36-a953-43dd-add8-dbe1ba7ce747" (UID: "86389b36-a953-43dd-add8-dbe1ba7ce747"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:32:39.397946 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.397923    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "86389b36-a953-43dd-add8-dbe1ba7ce747" (UID: "86389b36-a953-43dd-add8-dbe1ba7ce747"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:32:39.398973 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.398950    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-token" (OuterVolumeSpecName: "istio-token") pod "86389b36-a953-43dd-add8-dbe1ba7ce747" (UID: "86389b36-a953-43dd-add8-dbe1ba7ce747"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:32:39.495182 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.495145    2578 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-kubeconfig\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:32:39.495182 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.495173    2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-token\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:32:39.495182 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.495184    2578 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-cacerts\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:32:39.495478 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.495196    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-ca-configmap\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:32:39.495478 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.495206    2578 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/86389b36-a953-43dd-add8-dbe1ba7ce747-istio-csr-dns-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:32:39.495478 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.495215    2578 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/86389b36-a953-43dd-add8-dbe1ba7ce747-local-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:32:39.495478 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.495225    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5k7xn\" (UniqueName: \"kubernetes.io/projected/86389b36-a953-43dd-add8-dbe1ba7ce747-kube-api-access-5k7xn\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:32:39.502311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.502272    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns" event={"ID":"a9446da5-a76a-4045-b2f7-b6461c1fb59d","Type":"ContainerStarted","Data":"061497f86f55fe1049dd34322dacb36cf97761e70025895f3431954a7ec71d16"}
Sep 10 14:32:39.502489 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.502339    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:32:39.504502 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.504472    2578 generic.go:356] "Generic (PLEG): container finished" podID="86389b36-a953-43dd-add8-dbe1ba7ce747" containerID="df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159" exitCode=0
Sep 10 14:32:39.504622 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.504542    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k" event={"ID":"86389b36-a953-43dd-add8-dbe1ba7ce747","Type":"ContainerDied","Data":"df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159"}
Sep 10 14:32:39.504622 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.504576    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k" event={"ID":"86389b36-a953-43dd-add8-dbe1ba7ce747","Type":"ContainerDied","Data":"87bc25ca42d33f57b64408651dbf742a627888fbcac8e79a72a5fac3a32d4712"}
Sep 10 14:32:39.504811 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.504794    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"
Sep 10 14:32:39.505434 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.504600    2578 scope.go:117] "RemoveContainer" containerID="df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159"
Sep 10 14:32:39.521780 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.521720    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns" podStartSLOduration=1.858828818 podStartE2EDuration="4.521704431s" totalImagesPullingTime="2.662875613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:39.520030616 +0000 UTC m=+765.287753398" watchObservedRunningTime="2026-09-10 14:32:39.521704431 +0000 UTC m=+765.289427212"
Sep 10 14:32:39.539497 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.539474    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"]
Sep 10 14:32:39.545000 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.544978    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-49x7k"]
Sep 10 14:32:39.924476 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.924339    2578 scope.go:117] "RemoveContainer" containerID="df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159"
Sep 10 14:32:39.924758 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:32:39.924724    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159\": container with ID starting with df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159 not found: ID does not exist" containerID="df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159"
Sep 10 14:32:39.924807 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:39.924780    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159"} err="failed to get container status \"df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159\": rpc error: code = NotFound desc = could not find container \"df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159\": container with ID starting with df71426805a8e32ff2bdb41a8560ea007aea0cd9e4beebfdcaf9514c7d724159 not found: ID does not exist"
Sep 10 14:32:40.510458 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:40.510418    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4" event={"ID":"37d62584-d7e1-4aed-9fcd-37ffbeaa687a","Type":"ContainerStarted","Data":"98768608e9da0ed90b917a6a17f6325aecbc75a12468e91aa1b840f12304eb91"}
Sep 10 14:32:40.510671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:40.510575    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:32:40.528258 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:40.528211    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4" podStartSLOduration=2.7080904930000003 podStartE2EDuration="5.528193601s" totalImagesPullingTime="2.820103108s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:32:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:32:40.52779871 +0000 UTC m=+766.295521492" watchObservedRunningTime="2026-09-10 14:32:40.528193601 +0000 UTC m=+766.295916381"
Sep 10 14:32:40.723463 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:32:40.723430    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86389b36-a953-43dd-add8-dbe1ba7ce747" path="/var/lib/kubelet/pods/86389b36-a953-43dd-add8-dbe1ba7ce747/volumes"
Sep 10 14:33:10.517405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:10.517371    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-mnjns"
Sep 10 14:33:11.518404 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:11.518370    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:33:26.945185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:26.945144    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:27.042887 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:27.042839    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:43.529530 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:43.529491    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:54.105730 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:54.105698    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:55.077019 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:55.076979    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:57.408260 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.408224    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-j8glg"]
Sep 10 14:33:57.408644 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.408530    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86389b36-a953-43dd-add8-dbe1ba7ce747" containerName="discovery"
Sep 10 14:33:57.408644 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.408541    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="86389b36-a953-43dd-add8-dbe1ba7ce747" containerName="discovery"
Sep 10 14:33:57.408644 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.408588    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="86389b36-a953-43dd-add8-dbe1ba7ce747" containerName="discovery"
Sep 10 14:33:57.411505 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.411482    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-j8glg"
Sep 10 14:33:57.413497 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.413464    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-z2g77\""
Sep 10 14:33:57.413616 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.413550    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 10 14:33:57.413847 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.413835    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 10 14:33:57.423447 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.423425    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-j8glg"]
Sep 10 14:33:57.458415 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.458374    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v74h8\" (UniqueName: \"kubernetes.io/projected/54ac7e88-70f7-4bfc-a6a7-7c882ca93715-kube-api-access-v74h8\") pod \"authorino-operator-76b478bd78-j8glg\" (UID: \"54ac7e88-70f7-4bfc-a6a7-7c882ca93715\") " pod="kuadrant-system/authorino-operator-76b478bd78-j8glg"
Sep 10 14:33:57.559600 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.559556    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v74h8\" (UniqueName: \"kubernetes.io/projected/54ac7e88-70f7-4bfc-a6a7-7c882ca93715-kube-api-access-v74h8\") pod \"authorino-operator-76b478bd78-j8glg\" (UID: \"54ac7e88-70f7-4bfc-a6a7-7c882ca93715\") " pod="kuadrant-system/authorino-operator-76b478bd78-j8glg"
Sep 10 14:33:57.580836 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.580797    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v74h8\" (UniqueName: \"kubernetes.io/projected/54ac7e88-70f7-4bfc-a6a7-7c882ca93715-kube-api-access-v74h8\") pod \"authorino-operator-76b478bd78-j8glg\" (UID: \"54ac7e88-70f7-4bfc-a6a7-7c882ca93715\") " pod="kuadrant-system/authorino-operator-76b478bd78-j8glg"
Sep 10 14:33:57.721851 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:57.721722    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-j8glg"
Sep 10 14:33:58.009645 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:58.009567    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-j8glg"]
Sep 10 14:33:58.012736 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:33:58.012699    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54ac7e88_70f7_4bfc_a6a7_7c882ca93715.slice/crio-2b44537b7a10445bd7325b9f233f95718c1aa5c1de1c74621050b340af8cacd0 WatchSource:0}: Error finding container 2b44537b7a10445bd7325b9f233f95718c1aa5c1de1c74621050b340af8cacd0: Status 404 returned error can't find the container with id 2b44537b7a10445bd7325b9f233f95718c1aa5c1de1c74621050b340af8cacd0
Sep 10 14:33:58.033204 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:58.033170    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:58.186691 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:58.186650    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:58.343293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:58.343252    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:33:58.838592 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:58.838554    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-j8glg" event={"ID":"54ac7e88-70f7-4bfc-a6a7-7c882ca93715","Type":"ContainerStarted","Data":"2b44537b7a10445bd7325b9f233f95718c1aa5c1de1c74621050b340af8cacd0"}
Sep 10 14:33:58.920509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:33:58.920478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:00.654652 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:00.654609    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:00.754022 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:00.753983    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:01.820817 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.820780    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"]
Sep 10 14:34:01.825276 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.825255    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:01.827418 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.827393    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8tcdd\""
Sep 10 14:34:01.841082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.841048    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"]
Sep 10 14:34:01.895590 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.895544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzq2z\" (UniqueName: \"kubernetes.io/projected/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-kube-api-access-gzq2z\") pod \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" (UID: \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:01.895819 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.895610    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" (UID: \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:01.996809 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.996768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" (UID: \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:01.997009 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.996879    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzq2z\" (UniqueName: \"kubernetes.io/projected/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-kube-api-access-gzq2z\") pod \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" (UID: \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:01.997182 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:01.997159    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" (UID: \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:02.005978 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:02.005955    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzq2z\" (UniqueName: \"kubernetes.io/projected/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-kube-api-access-gzq2z\") pod \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" (UID: \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:02.135680 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:02.135580    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:02.342990 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:02.342956    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:02.419112 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:02.419089    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"]
Sep 10 14:34:02.421642 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:34:02.421617    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded098803_ff62_48d6_b2ba_c6a5c4f0895b.slice/crio-8d91982227c2d8ad627cbaa95bc855197142454dd4fb034899cd53a3a96c9c36 WatchSource:0}: Error finding container 8d91982227c2d8ad627cbaa95bc855197142454dd4fb034899cd53a3a96c9c36: Status 404 returned error can't find the container with id 8d91982227c2d8ad627cbaa95bc855197142454dd4fb034899cd53a3a96c9c36
Sep 10 14:34:02.566767 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:02.566714    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:02.858382 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:02.858342    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b" event={"ID":"ed098803-ff62-48d6-b2ba-c6a5c4f0895b","Type":"ContainerStarted","Data":"8d91982227c2d8ad627cbaa95bc855197142454dd4fb034899cd53a3a96c9c36"}
Sep 10 14:34:03.973938 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:03.972350    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"]
Sep 10 14:34:03.976406 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:03.976056    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"
Sep 10 14:34:03.979222 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:03.979191    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 10 14:34:03.979727 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:03.979552    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-rqfp5\""
Sep 10 14:34:03.988691 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:03.988634    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"]
Sep 10 14:34:04.016176 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:04.016136    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5ptvn\" (UniqueName: \"kubernetes.io/projected/54745ef5-fbf2-46c8-afd1-407f20e7c1c0-kube-api-access-5ptvn\") pod \"dns-operator-controller-manager-7b84cd7c9c-skwbf\" (UID: \"54745ef5-fbf2-46c8-afd1-407f20e7c1c0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"
Sep 10 14:34:04.116928 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:04.116869    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5ptvn\" (UniqueName: \"kubernetes.io/projected/54745ef5-fbf2-46c8-afd1-407f20e7c1c0-kube-api-access-5ptvn\") pod \"dns-operator-controller-manager-7b84cd7c9c-skwbf\" (UID: \"54745ef5-fbf2-46c8-afd1-407f20e7c1c0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"
Sep 10 14:34:04.130764 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:04.130649    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5ptvn\" (UniqueName: \"kubernetes.io/projected/54745ef5-fbf2-46c8-afd1-407f20e7c1c0-kube-api-access-5ptvn\") pod \"dns-operator-controller-manager-7b84cd7c9c-skwbf\" (UID: \"54745ef5-fbf2-46c8-afd1-407f20e7c1c0\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"
Sep 10 14:34:04.292643 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:04.292488    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"
Sep 10 14:34:04.512045 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:04.511989    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:06.102460 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.102417    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"]
Sep 10 14:34:06.106263 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.106233    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:06.108427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.108398    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-7lxlr\""
Sep 10 14:34:06.115193 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.115024    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"]
Sep 10 14:34:06.138941 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.138900    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq4kv\" (UniqueName: \"kubernetes.io/projected/c7245ab4-c17b-441e-92e6-11b13f39fc45-kube-api-access-wq4kv\") pod \"limitador-operator-controller-manager-74495c69f8-jvfq9\" (UID: \"c7245ab4-c17b-441e-92e6-11b13f39fc45\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:06.240005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.239965    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq4kv\" (UniqueName: \"kubernetes.io/projected/c7245ab4-c17b-441e-92e6-11b13f39fc45-kube-api-access-wq4kv\") pod \"limitador-operator-controller-manager-74495c69f8-jvfq9\" (UID: \"c7245ab4-c17b-441e-92e6-11b13f39fc45\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:06.251839 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.251801    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq4kv\" (UniqueName: \"kubernetes.io/projected/c7245ab4-c17b-441e-92e6-11b13f39fc45-kube-api-access-wq4kv\") pod \"limitador-operator-controller-manager-74495c69f8-jvfq9\" (UID: \"c7245ab4-c17b-441e-92e6-11b13f39fc45\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:06.420999 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.420897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:06.578447 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.578413    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:06.883572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.883534    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-j8glg" event={"ID":"54ac7e88-70f7-4bfc-a6a7-7c882ca93715","Type":"ContainerStarted","Data":"babdf908d868c3bb50a9ad338fb814981c89c7cc7045b6b2fae1245190e3be9a"}
Sep 10 14:34:06.883770 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.883642    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-j8glg"
Sep 10 14:34:06.885372 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.885346    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b" event={"ID":"ed098803-ff62-48d6-b2ba-c6a5c4f0895b","Type":"ContainerStarted","Data":"7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa"}
Sep 10 14:34:06.885491 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.885465    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:06.940439 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.940318    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b" podStartSLOduration=1.645991153 podStartE2EDuration="5.940295015s" totalImagesPullingTime="4.294303862s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:06.939709035 +0000 UTC m=+852.707431817" watchObservedRunningTime="2026-09-10 14:34:06.940295015 +0000 UTC m=+852.708017797"
Sep 10 14:34:06.941080 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.941043    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-j8glg" podStartSLOduration=1.25801723 podStartE2EDuration="9.941031231s" totalImagesPullingTime="8.683014001s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:33:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:06.91311882 +0000 UTC m=+852.680841600" watchObservedRunningTime="2026-09-10 14:34:06.941031231 +0000 UTC m=+852.708754014"
Sep 10 14:34:06.971778 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.971724    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"]
Sep 10 14:34:06.975455 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:34:06.975416    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54745ef5_fbf2_46c8_afd1_407f20e7c1c0.slice/crio-46dc1edf9ef9fde7becb059726b0b9a658eba6b0833eae9f3f49013ce8df9b07 WatchSource:0}: Error finding container 46dc1edf9ef9fde7becb059726b0b9a658eba6b0833eae9f3f49013ce8df9b07: Status 404 returned error can't find the container with id 46dc1edf9ef9fde7becb059726b0b9a658eba6b0833eae9f3f49013ce8df9b07
Sep 10 14:34:06.985075 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:06.985053    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"]
Sep 10 14:34:06.987121 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:34:06.987090    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7245ab4_c17b_441e_92e6_11b13f39fc45.slice/crio-9a312a48d27d24f955ce116b4b58a5e8a864b6d63903c9541e1bedf94b53b5ac WatchSource:0}: Error finding container 9a312a48d27d24f955ce116b4b58a5e8a864b6d63903c9541e1bedf94b53b5ac: Status 404 returned error can't find the container with id 9a312a48d27d24f955ce116b4b58a5e8a864b6d63903c9541e1bedf94b53b5ac
Sep 10 14:34:07.400636 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:07.400600    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:07.511367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:07.511329    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:07.735973 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:07.735875    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:07.838767 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:07.838694    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:07.891983 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:07.891942    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf" event={"ID":"54745ef5-fbf2-46c8-afd1-407f20e7c1c0","Type":"ContainerStarted","Data":"46dc1edf9ef9fde7becb059726b0b9a658eba6b0833eae9f3f49013ce8df9b07"}
Sep 10 14:34:07.893489 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:07.893459    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9" event={"ID":"c7245ab4-c17b-441e-92e6-11b13f39fc45","Type":"ContainerStarted","Data":"9a312a48d27d24f955ce116b4b58a5e8a864b6d63903c9541e1bedf94b53b5ac"}
Sep 10 14:34:09.906687 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.906643    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf" event={"ID":"54745ef5-fbf2-46c8-afd1-407f20e7c1c0","Type":"ContainerStarted","Data":"d02804ab5d509e95d1ba99ba6a0450c9ec1aa26ad704aad6bc2d1eb44ecb0bcc"}
Sep 10 14:34:09.907169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.906803    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"
Sep 10 14:34:09.908399 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.908377    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9" event={"ID":"c7245ab4-c17b-441e-92e6-11b13f39fc45","Type":"ContainerStarted","Data":"be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39"}
Sep 10 14:34:09.908530 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.908501    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:09.927374 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.927321    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf" podStartSLOduration=4.793824247 podStartE2EDuration="6.927304567s" totalImagesPullingTime="2.13348032s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:09.925780443 +0000 UTC m=+855.693503225" watchObservedRunningTime="2026-09-10 14:34:09.927304567 +0000 UTC m=+855.695027348"
Sep 10 14:34:09.930942 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.930918    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"]
Sep 10 14:34:09.934135 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.934116    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:09.936203 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.936103    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 10 14:34:09.936203 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.936139    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 10 14:34:09.936370 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.936354    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dmbns\""
Sep 10 14:34:09.940188 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.940168    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"]
Sep 10 14:34:09.945044 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.944919    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9" podStartSLOduration=1.877334981 podStartE2EDuration="3.944888849s" totalImagesPullingTime="2.067553868s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:09.944302344 +0000 UTC m=+855.712025126" watchObservedRunningTime="2026-09-10 14:34:09.944888849 +0000 UTC m=+855.712611630"
Sep 10 14:34:09.947160 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.947136    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:09.975527 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.975495    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6nzf\" (UniqueName: \"kubernetes.io/projected/75256127-4d2c-4928-a207-07717cfb6c86-kube-api-access-p6nzf\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:09.975682 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.975553    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/75256127-4d2c-4928-a207-07717cfb6c86-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:09.975803 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:09.975774    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/75256127-4d2c-4928-a207-07717cfb6c86-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:10.040684 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.040642    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:10.077168 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.077124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/75256127-4d2c-4928-a207-07717cfb6c86-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:10.077341 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.077172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6nzf\" (UniqueName: \"kubernetes.io/projected/75256127-4d2c-4928-a207-07717cfb6c86-kube-api-access-p6nzf\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:10.077341 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.077221    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/75256127-4d2c-4928-a207-07717cfb6c86-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:10.077962 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.077943    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/75256127-4d2c-4928-a207-07717cfb6c86-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:10.079787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.079739    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/75256127-4d2c-4928-a207-07717cfb6c86-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:10.085040 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.085017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6nzf\" (UniqueName: \"kubernetes.io/projected/75256127-4d2c-4928-a207-07717cfb6c86-kube-api-access-p6nzf\") pod \"kuadrant-console-plugin-fbdd4f445-zckzk\" (UID: \"75256127-4d2c-4928-a207-07717cfb6c86\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:10.244787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.244671    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"
Sep 10 14:34:10.519465 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.519407    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk"]
Sep 10 14:34:10.522075 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:34:10.522050    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75256127_4d2c_4928_a207_07717cfb6c86.slice/crio-2ec72174f625225c1d80de7a5e46c39c1810938019404118028b713957dab89e WatchSource:0}: Error finding container 2ec72174f625225c1d80de7a5e46c39c1810938019404118028b713957dab89e: Status 404 returned error can't find the container with id 2ec72174f625225c1d80de7a5e46c39c1810938019404118028b713957dab89e
Sep 10 14:34:10.914546 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:10.914503    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk" event={"ID":"75256127-4d2c-4928-a207-07717cfb6c86","Type":"ContainerStarted","Data":"2ec72174f625225c1d80de7a5e46c39c1810938019404118028b713957dab89e"}
Sep 10 14:34:12.126175 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:12.126120    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:12.185361 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:12.185318    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:12.522427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:12.522319    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:12.589845 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:12.589800    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:12.740325 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:12.740290    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:13.107483 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:13.107432    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:15.942335 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:15.942296    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk" event={"ID":"75256127-4d2c-4928-a207-07717cfb6c86","Type":"ContainerStarted","Data":"3e10a984ba2f6ff5349ff1b46c5329e3c2c08b720d8c6005434ad4949751cf69"}
Sep 10 14:34:15.961759 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:15.961685    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-zckzk" podStartSLOduration=2.334165143 podStartE2EDuration="6.961669155s" totalImagesPullingTime="4.627504012s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:15.960186318 +0000 UTC m=+861.727909099" watchObservedRunningTime="2026-09-10 14:34:15.961669155 +0000 UTC m=+861.729391935"
Sep 10 14:34:17.896075 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:17.896036    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-j8glg"
Sep 10 14:34:17.896471 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:17.896262    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:18.434238 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:18.434204    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:18.540941 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:18.540908    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:19.681011 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.680974    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"]
Sep 10 14:34:19.684269 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.684246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:19.699374 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.699345    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"]
Sep 10 14:34:19.763539 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.763508    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/114a5093-c715-425b-823f-04b0bc9faa30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" (UID: \"114a5093-c715-425b-823f-04b0bc9faa30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:19.763720 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.763546    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xwqsg\" (UniqueName: \"kubernetes.io/projected/114a5093-c715-425b-823f-04b0bc9faa30-kube-api-access-xwqsg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" (UID: \"114a5093-c715-425b-823f-04b0bc9faa30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:19.864707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.864674    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/114a5093-c715-425b-823f-04b0bc9faa30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" (UID: \"114a5093-c715-425b-823f-04b0bc9faa30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:19.864707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.864709    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xwqsg\" (UniqueName: \"kubernetes.io/projected/114a5093-c715-425b-823f-04b0bc9faa30-kube-api-access-xwqsg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" (UID: \"114a5093-c715-425b-823f-04b0bc9faa30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:19.865067 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.865049    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/114a5093-c715-425b-823f-04b0bc9faa30-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" (UID: \"114a5093-c715-425b-823f-04b0bc9faa30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:19.877090 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.877058    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xwqsg\" (UniqueName: \"kubernetes.io/projected/114a5093-c715-425b-823f-04b0bc9faa30-kube-api-access-xwqsg\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" (UID: \"114a5093-c715-425b-823f-04b0bc9faa30\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:19.995030 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:19.994938    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:20.110543 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.110501    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"]
Sep 10 14:34:20.110727 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.110696    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" containerName="manager" containerID="cri-o://7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa" gracePeriod=2
Sep 10 14:34:20.121999 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.121966    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"]
Sep 10 14:34:20.123646 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.123600    2578 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-76f8f686bf-92j2b\": pods \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:20.136722 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.136690    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"]
Sep 10 14:34:20.139824 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.139799    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"]
Sep 10 14:34:20.140156 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.140140    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" containerName="manager"
Sep 10 14:34:20.140156 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.140156    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" containerName="manager"
Sep 10 14:34:20.140285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.140241    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" containerName="manager"
Sep 10 14:34:20.143128 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.143104    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.145192 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.145158    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:20.148640 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.148615    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"]
Sep 10 14:34:20.163522 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.163487    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"]
Sep 10 14:34:20.167802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.167770    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"]
Sep 10 14:34:20.168013 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.167988    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" containerName="manager" containerID="cri-o://be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39" gracePeriod=2
Sep 10 14:34:20.170282 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.170237    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:20.183909 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.183878    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"]
Sep 10 14:34:20.187315 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.187289    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.198437 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.198400    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"]
Sep 10 14:34:20.206075 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.206043    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"]
Sep 10 14:34:20.210441 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.210404    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"]
Sep 10 14:34:20.210739 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.210727    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" containerName="manager"
Sep 10 14:34:20.210814 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.210740    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" containerName="manager"
Sep 10 14:34:20.210853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.210837    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" containerName="manager"
Sep 10 14:34:20.213634 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.213617    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"
Sep 10 14:34:20.237161 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.237127    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:20.240458 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.240421    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-jvfq9\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:20.245821 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.245686    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"]
Sep 10 14:34:20.268489 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.268432    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40477f8a-3733-461d-bcba-ec16e2205507-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-npj8z\" (UID: \"40477f8a-3733-461d-bcba-ec16e2205507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.268671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.268498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kstn2\" (UniqueName: \"kubernetes.io/projected/7603262d-213b-4732-9f82-238152e59023-kube-api-access-kstn2\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-q7plb\" (UID: \"7603262d-213b-4732-9f82-238152e59023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.268671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.268544    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjx44\" (UniqueName: \"kubernetes.io/projected/40477f8a-3733-461d-bcba-ec16e2205507-kube-api-access-hjx44\") pod \"kuadrant-operator-controller-manager-76f8f686bf-npj8z\" (UID: \"40477f8a-3733-461d-bcba-ec16e2205507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.268671 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.268603    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7603262d-213b-4732-9f82-238152e59023-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-q7plb\" (UID: \"7603262d-213b-4732-9f82-238152e59023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.268906 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.268729    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gvlfc\" (UniqueName: \"kubernetes.io/projected/b2f856a7-95c5-4e6c-b07f-59acdb3b3b15-kube-api-access-gvlfc\") pod \"limitador-operator-controller-manager-74495c69f8-mpn9t\" (UID: \"b2f856a7-95c5-4e6c-b07f-59acdb3b3b15\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"
Sep 10 14:34:20.283883 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.283839    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:20.285614 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.285578    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-jvfq9\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:20.369887 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.369850    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjx44\" (UniqueName: \"kubernetes.io/projected/40477f8a-3733-461d-bcba-ec16e2205507-kube-api-access-hjx44\") pod \"kuadrant-operator-controller-manager-76f8f686bf-npj8z\" (UID: \"40477f8a-3733-461d-bcba-ec16e2205507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.370061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.369921    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7603262d-213b-4732-9f82-238152e59023-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-q7plb\" (UID: \"7603262d-213b-4732-9f82-238152e59023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.370061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.369950    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gvlfc\" (UniqueName: \"kubernetes.io/projected/b2f856a7-95c5-4e6c-b07f-59acdb3b3b15-kube-api-access-gvlfc\") pod \"limitador-operator-controller-manager-74495c69f8-mpn9t\" (UID: \"b2f856a7-95c5-4e6c-b07f-59acdb3b3b15\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"
Sep 10 14:34:20.370061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.370016    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40477f8a-3733-461d-bcba-ec16e2205507-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-npj8z\" (UID: \"40477f8a-3733-461d-bcba-ec16e2205507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.370061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.370041    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kstn2\" (UniqueName: \"kubernetes.io/projected/7603262d-213b-4732-9f82-238152e59023-kube-api-access-kstn2\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-q7plb\" (UID: \"7603262d-213b-4732-9f82-238152e59023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.370426 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.370395    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7603262d-213b-4732-9f82-238152e59023-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-q7plb\" (UID: \"7603262d-213b-4732-9f82-238152e59023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.370619 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.370596    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40477f8a-3733-461d-bcba-ec16e2205507-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-npj8z\" (UID: \"40477f8a-3733-461d-bcba-ec16e2205507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.383590 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.382068    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kstn2\" (UniqueName: \"kubernetes.io/projected/7603262d-213b-4732-9f82-238152e59023-kube-api-access-kstn2\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-q7plb\" (UID: \"7603262d-213b-4732-9f82-238152e59023\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.383590 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.382151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjx44\" (UniqueName: \"kubernetes.io/projected/40477f8a-3733-461d-bcba-ec16e2205507-kube-api-access-hjx44\") pod \"kuadrant-operator-controller-manager-76f8f686bf-npj8z\" (UID: \"40477f8a-3733-461d-bcba-ec16e2205507\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.392842 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.392805    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gvlfc\" (UniqueName: \"kubernetes.io/projected/b2f856a7-95c5-4e6c-b07f-59acdb3b3b15-kube-api-access-gvlfc\") pod \"limitador-operator-controller-manager-74495c69f8-mpn9t\" (UID: \"b2f856a7-95c5-4e6c-b07f-59acdb3b3b15\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"
Sep 10 14:34:20.419610 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.419578    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:20.421779 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.421726    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:20.423402 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.423365    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-jvfq9\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:20.428300 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.428275    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:20.430942 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.430917    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-jvfq9\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:20.433048 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.433015    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-92j2b\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:20.438040 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.438007    2578 log.go:32] "RunPodSandbox from runtime service failed" err=<
Sep 10 14:34:20.438040 ip-10-0-142-68 kubenswrapper[2578]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-8wwpx_kuadrant-system_114a5093-c715-425b-823f-04b0bc9faa30_0(0b526238cab29c8ce0884ccffa54570fe82626189e0df4523ea7e89dc1e82d3f): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-8wwpx to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"0b526238cab29c8ce0884ccffa54570fe82626189e0df4523ea7e89dc1e82d3f" Netns:"/var/run/netns/0c108568-1cb1-4dc0-b4fe-68ae1eae3272" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-8wwpx;K8S_POD_INFRA_CONTAINER_ID=0b526238cab29c8ce0884ccffa54570fe82626189e0df4523ea7e89dc1e82d3f;K8S_POD_UID=114a5093-c715-425b-823f-04b0bc9faa30" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx/114a5093-c715-425b-823f-04b0bc9faa30:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx 0b526238cab29c8ce0884ccffa54570fe82626189e0df4523ea7e89dc1e82d3f network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:1d [10.134.0.29/23]
Sep 10 14:34:20.438040 ip-10-0-142-68 kubenswrapper[2578]:         '
Sep 10 14:34:20.438040 ip-10-0-142-68 kubenswrapper[2578]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 10 14:34:20.438040 ip-10-0-142-68 kubenswrapper[2578]:  >
Sep 10 14:34:20.438268 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.438063    2578 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err=<
Sep 10 14:34:20.438268 ip-10-0-142-68 kubenswrapper[2578]:         rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-7978cbdfbc-8wwpx_kuadrant-system_114a5093-c715-425b-823f-04b0bc9faa30_0(0b526238cab29c8ce0884ccffa54570fe82626189e0df4523ea7e89dc1e82d3f): error adding pod kuadrant-system_kuadrant-operator-controller-manager-7978cbdfbc-8wwpx to CNI network "multus-cni-network": plugin type="multus-shim" name="multus-cni-network" failed (add): CmdAdd (shim): CNI request failed with status 400: 'ContainerID:"0b526238cab29c8ce0884ccffa54570fe82626189e0df4523ea7e89dc1e82d3f" Netns:"/var/run/netns/0c108568-1cb1-4dc0-b4fe-68ae1eae3272" IfName:"eth0" Args:"IgnoreUnknown=1;K8S_POD_NAMESPACE=kuadrant-system;K8S_POD_NAME=kuadrant-operator-controller-manager-7978cbdfbc-8wwpx;K8S_POD_INFRA_CONTAINER_ID=0b526238cab29c8ce0884ccffa54570fe82626189e0df4523ea7e89dc1e82d3f;K8S_POD_UID=114a5093-c715-425b-823f-04b0bc9faa30" Path:"" ERRORED: error configuring pod [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx] networking: [kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx/114a5093-c715-425b-823f-04b0bc9faa30:ovn-kubernetes]: error adding container to network "ovn-kubernetes": CNI request failed with status 400: '[kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx 0b526238cab29c8ce0884ccffa54570fe82626189e0df4523ea7e89dc1e82d3f network default NAD default NAD key default] failed to configure pod interface: pod deleted waiting for OVS port binding for 0a:58:0a:86:00:1d [10.134.0.29/23]
Sep 10 14:34:20.438268 ip-10-0-142-68 kubenswrapper[2578]:         '
Sep 10 14:34:20.438268 ip-10-0-142-68 kubenswrapper[2578]:         ': StdinData: {"auxiliaryCNIChainName":"vendor-cni-chain","binDir":"/var/lib/cni/bin","clusterNetwork":"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf","cniVersion":"0.3.1","daemonSocketDir":"/run/multus/socket","globalNamespaces":"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv","logLevel":"verbose","logToStderr":true,"name":"multus-cni-network","namespaceIsolation":true,"type":"multus-shim"}
Sep 10 14:34:20.438268 ip-10-0-142-68 kubenswrapper[2578]:  > pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:20.545936 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.545844    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.553105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.553080    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.559114 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.559088    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"
Sep 10 14:34:20.572608 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.572575    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wq4kv\" (UniqueName: \"kubernetes.io/projected/c7245ab4-c17b-441e-92e6-11b13f39fc45-kube-api-access-wq4kv\") pod \"c7245ab4-c17b-441e-92e6-11b13f39fc45\" (UID: \"c7245ab4-c17b-441e-92e6-11b13f39fc45\") "
Sep 10 14:34:20.572795 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.572650    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-extensions-socket-volume\") pod \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\" (UID: \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\") "
Sep 10 14:34:20.572795 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.572696    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gzq2z\" (UniqueName: \"kubernetes.io/projected/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-kube-api-access-gzq2z\") pod \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\" (UID: \"ed098803-ff62-48d6-b2ba-c6a5c4f0895b\") "
Sep 10 14:34:20.573398 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.573370    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ed098803-ff62-48d6-b2ba-c6a5c4f0895b" (UID: "ed098803-ff62-48d6-b2ba-c6a5c4f0895b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:20.575094 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.575056    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7245ab4-c17b-441e-92e6-11b13f39fc45-kube-api-access-wq4kv" (OuterVolumeSpecName: "kube-api-access-wq4kv") pod "c7245ab4-c17b-441e-92e6-11b13f39fc45" (UID: "c7245ab4-c17b-441e-92e6-11b13f39fc45"). InnerVolumeSpecName "kube-api-access-wq4kv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:20.575533 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.575499    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-kube-api-access-gzq2z" (OuterVolumeSpecName: "kube-api-access-gzq2z") pod "ed098803-ff62-48d6-b2ba-c6a5c4f0895b" (UID: "ed098803-ff62-48d6-b2ba-c6a5c4f0895b"). InnerVolumeSpecName "kube-api-access-gzq2z". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:20.674296 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.674264    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wq4kv\" (UniqueName: \"kubernetes.io/projected/c7245ab4-c17b-441e-92e6-11b13f39fc45-kube-api-access-wq4kv\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:20.674296 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.674294    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-extensions-socket-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:20.674533 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.674327    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gzq2z\" (UniqueName: \"kubernetes.io/projected/ed098803-ff62-48d6-b2ba-c6a5c4f0895b-kube-api-access-gzq2z\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:20.723950 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.723917    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7245ab4-c17b-441e-92e6-11b13f39fc45" path="/var/lib/kubelet/pods/c7245ab4-c17b-441e-92e6-11b13f39fc45/volumes"
Sep 10 14:34:20.724312 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.724243    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" path="/var/lib/kubelet/pods/ed098803-ff62-48d6-b2ba-c6a5c4f0895b/volumes"
Sep 10 14:34:20.829246 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.829187    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"]
Sep 10 14:34:20.832201 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:34:20.832171    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40477f8a_3733_461d_bcba_ec16e2205507.slice/crio-ffb44df1f54e9bc3b728ef64335635e1006173d78da2bf7c36e3cffff703b3eb WatchSource:0}: Error finding container ffb44df1f54e9bc3b728ef64335635e1006173d78da2bf7c36e3cffff703b3eb: Status 404 returned error can't find the container with id ffb44df1f54e9bc3b728ef64335635e1006173d78da2bf7c36e3cffff703b3eb
Sep 10 14:34:20.850736 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.850649    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"]
Sep 10 14:34:20.852311 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:34:20.852287    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7603262d_213b_4732_9f82_238152e59023.slice/crio-36657374d4c1ad9e1acf28649691f6068d6594ecc0dba07d305849e3a7e6ab14 WatchSource:0}: Error finding container 36657374d4c1ad9e1acf28649691f6068d6594ecc0dba07d305849e3a7e6ab14: Status 404 returned error can't find the container with id 36657374d4c1ad9e1acf28649691f6068d6594ecc0dba07d305849e3a7e6ab14
Sep 10 14:34:20.863298 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.863273    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"]
Sep 10 14:34:20.868835 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:34:20.868808    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb2f856a7_95c5_4e6c_b07f_59acdb3b3b15.slice/crio-0166c2cff34ed6e832611eabfa87656d40a6e395c2fcf7bca59d64df28cd8aed WatchSource:0}: Error finding container 0166c2cff34ed6e832611eabfa87656d40a6e395c2fcf7bca59d64df28cd8aed: Status 404 returned error can't find the container with id 0166c2cff34ed6e832611eabfa87656d40a6e395c2fcf7bca59d64df28cd8aed
Sep 10 14:34:20.917151 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.917128    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-skwbf"
Sep 10 14:34:20.969986 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.969950    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb" event={"ID":"7603262d-213b-4732-9f82-238152e59023","Type":"ContainerStarted","Data":"4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a"}
Sep 10 14:34:20.970212 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.970000    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb" event={"ID":"7603262d-213b-4732-9f82-238152e59023","Type":"ContainerStarted","Data":"36657374d4c1ad9e1acf28649691f6068d6594ecc0dba07d305849e3a7e6ab14"}
Sep 10 14:34:20.970212 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.970042    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:20.972223 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.972195    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z" event={"ID":"40477f8a-3733-461d-bcba-ec16e2205507","Type":"ContainerStarted","Data":"2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8"}
Sep 10 14:34:20.972344 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.972247    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z" event={"ID":"40477f8a-3733-461d-bcba-ec16e2205507","Type":"ContainerStarted","Data":"ffb44df1f54e9bc3b728ef64335635e1006173d78da2bf7c36e3cffff703b3eb"}
Sep 10 14:34:20.972344 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.972278    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:20.974009 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.973988    2578 generic.go:356] "Generic (PLEG): container finished" podID="ed098803-ff62-48d6-b2ba-c6a5c4f0895b" containerID="7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa" exitCode=0
Sep 10 14:34:20.974120 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.974079    2578 scope.go:117] "RemoveContainer" containerID="7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa"
Sep 10 14:34:20.974211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.974197    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-92j2b"
Sep 10 14:34:20.978031 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.977982    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t" event={"ID":"b2f856a7-95c5-4e6c-b07f-59acdb3b3b15","Type":"ContainerStarted","Data":"8a81475be9fc100a8baa86162ddf1cdd11d1bd64ad0b24ae15489217bdcec651"}
Sep 10 14:34:20.978155 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.978045    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"
Sep 10 14:34:20.978155 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.978071    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t" event={"ID":"b2f856a7-95c5-4e6c-b07f-59acdb3b3b15","Type":"ContainerStarted","Data":"0166c2cff34ed6e832611eabfa87656d40a6e395c2fcf7bca59d64df28cd8aed"}
Sep 10 14:34:20.979832 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.979810    2578 generic.go:356] "Generic (PLEG): container finished" podID="c7245ab4-c17b-441e-92e6-11b13f39fc45" containerID="be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39" exitCode=0
Sep 10 14:34:20.979992 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.979883    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:20.980344 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.980324    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-jvfq9"
Sep 10 14:34:20.988425 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.988409    2578 scope.go:117] "RemoveContainer" containerID="7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa"
Sep 10 14:34:20.988706 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.988686    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa\": container with ID starting with 7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa not found: ID does not exist" containerID="7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa"
Sep 10 14:34:20.988813 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.988713    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa"} err="failed to get container status \"7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa\": rpc error: code = NotFound desc = could not find container \"7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa\": container with ID starting with 7625986527d002dc220075a05dda6eb8beecbccb5e93bad8f78b71557eeff7fa not found: ID does not exist"
Sep 10 14:34:20.988813 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.988731    2578 scope.go:117] "RemoveContainer" containerID="be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39"
Sep 10 14:34:20.989487 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.989469    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:20.996470 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.996455    2578 scope.go:117] "RemoveContainer" containerID="be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39"
Sep 10 14:34:20.996777 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:20.996725    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39\": container with ID starting with be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39 not found: ID does not exist" containerID="be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39"
Sep 10 14:34:20.996777 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:20.996765    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39"} err="failed to get container status \"be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39\": rpc error: code = NotFound desc = could not find container \"be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39\": container with ID starting with be48e9137c0b8ff72aea9635a458daa42dd1887a6aa1f96a9a8e6c4937801e39 not found: ID does not exist"
Sep 10 14:34:21.000582 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.000537    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb" podStartSLOduration=1.00052379 podStartE2EDuration="1.00052379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:20.99972667 +0000 UTC m=+866.767449451" watchObservedRunningTime="2026-09-10 14:34:21.00052379 +0000 UTC m=+866.768246572"
Sep 10 14:34:21.030246 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:21.030206    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="114a5093-c715-425b-823f-04b0bc9faa30" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:21.063838 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.063787    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z" podStartSLOduration=1.063771692 podStartE2EDuration="1.063771692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:21.063438305 +0000 UTC m=+866.831161085" watchObservedRunningTime="2026-09-10 14:34:21.063771692 +0000 UTC m=+866.831494472"
Sep 10 14:34:21.064394 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.064367    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t" podStartSLOduration=1.064359377 podStartE2EDuration="1.064359377s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:21.027496469 +0000 UTC m=+866.795219251" watchObservedRunningTime="2026-09-10 14:34:21.064359377 +0000 UTC m=+866.832082158"
Sep 10 14:34:21.077722 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.077686    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xwqsg\" (UniqueName: \"kubernetes.io/projected/114a5093-c715-425b-823f-04b0bc9faa30-kube-api-access-xwqsg\") pod \"114a5093-c715-425b-823f-04b0bc9faa30\" (UID: \"114a5093-c715-425b-823f-04b0bc9faa30\") "
Sep 10 14:34:21.077911 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.077841    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/114a5093-c715-425b-823f-04b0bc9faa30-extensions-socket-volume\") pod \"114a5093-c715-425b-823f-04b0bc9faa30\" (UID: \"114a5093-c715-425b-823f-04b0bc9faa30\") "
Sep 10 14:34:21.078465 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.078179    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/114a5093-c715-425b-823f-04b0bc9faa30-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "114a5093-c715-425b-823f-04b0bc9faa30" (UID: "114a5093-c715-425b-823f-04b0bc9faa30"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:21.080415 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.080353    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/114a5093-c715-425b-823f-04b0bc9faa30-kube-api-access-xwqsg" (OuterVolumeSpecName: "kube-api-access-xwqsg") pod "114a5093-c715-425b-823f-04b0bc9faa30" (UID: "114a5093-c715-425b-823f-04b0bc9faa30"). InnerVolumeSpecName "kube-api-access-xwqsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:21.178787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.178716    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/114a5093-c715-425b-823f-04b0bc9faa30-extensions-socket-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:21.178787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.178774    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xwqsg\" (UniqueName: \"kubernetes.io/projected/114a5093-c715-425b-823f-04b0bc9faa30-kube-api-access-xwqsg\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:21.987716 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:21.987677    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:21.989607 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:21.989578    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="114a5093-c715-425b-823f-04b0bc9faa30" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:21.994786 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:21.994727    2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-8wwpx\" is forbidden: User \"system:node:ip-10-0-142-68.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-68.ec2.internal' and this object" podUID="114a5093-c715-425b-823f-04b0bc9faa30" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-8wwpx"
Sep 10 14:34:22.724194 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:22.724158    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="114a5093-c715-425b-823f-04b0bc9faa30" path="/var/lib/kubelet/pods/114a5093-c715-425b-823f-04b0bc9faa30/volumes"
Sep 10 14:34:23.164412 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:23.164369    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:23.266264 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:23.266231    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:26.937605 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:26.937562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:27.007646 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:27.007606    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:27.143430 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:27.143395    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:27.245636 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:27.245537    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:31.165155 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:31.165116    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:31.232935 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:31.232903    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:31.989842 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:31.989812    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:31.990006 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:31.989884    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-mpn9t"
Sep 10 14:34:31.990048 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:31.990033    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:32.080069 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.080039    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"]
Sep 10 14:34:32.080254 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.080231    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z" podUID="40477f8a-3733-461d-bcba-ec16e2205507" containerName="manager" containerID="cri-o://2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8" gracePeriod=10
Sep 10 14:34:32.324882 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.324855    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:32.481386 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.481349    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjx44\" (UniqueName: \"kubernetes.io/projected/40477f8a-3733-461d-bcba-ec16e2205507-kube-api-access-hjx44\") pod \"40477f8a-3733-461d-bcba-ec16e2205507\" (UID: \"40477f8a-3733-461d-bcba-ec16e2205507\") "
Sep 10 14:34:32.481386 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.481412    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40477f8a-3733-461d-bcba-ec16e2205507-extensions-socket-volume\") pod \"40477f8a-3733-461d-bcba-ec16e2205507\" (UID: \"40477f8a-3733-461d-bcba-ec16e2205507\") "
Sep 10 14:34:32.481873 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.481845    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40477f8a-3733-461d-bcba-ec16e2205507-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "40477f8a-3733-461d-bcba-ec16e2205507" (UID: "40477f8a-3733-461d-bcba-ec16e2205507"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:32.483642 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.483609    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40477f8a-3733-461d-bcba-ec16e2205507-kube-api-access-hjx44" (OuterVolumeSpecName: "kube-api-access-hjx44") pod "40477f8a-3733-461d-bcba-ec16e2205507" (UID: "40477f8a-3733-461d-bcba-ec16e2205507"). InnerVolumeSpecName "kube-api-access-hjx44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:32.493399 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.493348    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"]
Sep 10 14:34:32.493710 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.493698    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40477f8a-3733-461d-bcba-ec16e2205507" containerName="manager"
Sep 10 14:34:32.493781 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.493712    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="40477f8a-3733-461d-bcba-ec16e2205507" containerName="manager"
Sep 10 14:34:32.493781 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.493779    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="40477f8a-3733-461d-bcba-ec16e2205507" containerName="manager"
Sep 10 14:34:32.497115 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.497093    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:32.519235 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.519212    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"]
Sep 10 14:34:32.581945 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.581906    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5lbk\" (UniqueName: \"kubernetes.io/projected/f24673a9-6616-43ae-9993-903cb25fe08b-kube-api-access-v5lbk\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bm9qx\" (UID: \"f24673a9-6616-43ae-9993-903cb25fe08b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:32.582096 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.582030    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24673a9-6616-43ae-9993-903cb25fe08b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bm9qx\" (UID: \"f24673a9-6616-43ae-9993-903cb25fe08b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:32.582169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.582101    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjx44\" (UniqueName: \"kubernetes.io/projected/40477f8a-3733-461d-bcba-ec16e2205507-kube-api-access-hjx44\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:32.582169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.582112    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/40477f8a-3733-461d-bcba-ec16e2205507-extensions-socket-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:32.682768 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.682700    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24673a9-6616-43ae-9993-903cb25fe08b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bm9qx\" (UID: \"f24673a9-6616-43ae-9993-903cb25fe08b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:32.682933 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.682786    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v5lbk\" (UniqueName: \"kubernetes.io/projected/f24673a9-6616-43ae-9993-903cb25fe08b-kube-api-access-v5lbk\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bm9qx\" (UID: \"f24673a9-6616-43ae-9993-903cb25fe08b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:32.683181 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.683156    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24673a9-6616-43ae-9993-903cb25fe08b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bm9qx\" (UID: \"f24673a9-6616-43ae-9993-903cb25fe08b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:32.691219 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.691188    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5lbk\" (UniqueName: \"kubernetes.io/projected/f24673a9-6616-43ae-9993-903cb25fe08b-kube-api-access-v5lbk\") pod \"kuadrant-operator-controller-manager-5769f99d9f-bm9qx\" (UID: \"f24673a9-6616-43ae-9993-903cb25fe08b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:32.806707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:32.806622    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:33.043319 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.043281    2578 generic.go:356] "Generic (PLEG): container finished" podID="40477f8a-3733-461d-bcba-ec16e2205507" containerID="2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8" exitCode=0
Sep 10 14:34:33.043514 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.043325    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"
Sep 10 14:34:33.043514 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.043342    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z" event={"ID":"40477f8a-3733-461d-bcba-ec16e2205507","Type":"ContainerDied","Data":"2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8"}
Sep 10 14:34:33.043514 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.043396    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z" event={"ID":"40477f8a-3733-461d-bcba-ec16e2205507","Type":"ContainerDied","Data":"ffb44df1f54e9bc3b728ef64335635e1006173d78da2bf7c36e3cffff703b3eb"}
Sep 10 14:34:33.043514 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.043424    2578 scope.go:117] "RemoveContainer" containerID="2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8"
Sep 10 14:34:33.052072 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.052052    2578 scope.go:117] "RemoveContainer" containerID="2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8"
Sep 10 14:34:33.052422 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:33.052394    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8\": container with ID starting with 2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8 not found: ID does not exist" containerID="2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8"
Sep 10 14:34:33.052473 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.052431    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8"} err="failed to get container status \"2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8\": rpc error: code = NotFound desc = could not find container \"2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8\": container with ID starting with 2f3ad5ea8e4c1701947ff43ad3bb5a4e0023e6062a29795c74761f4b84ac79b8 not found: ID does not exist"
Sep 10 14:34:33.062969 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.062887    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"]
Sep 10 14:34:33.070796 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.070762    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-npj8z"]
Sep 10 14:34:33.086834 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:33.086804    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"]
Sep 10 14:34:33.090523 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:34:33.090488    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf24673a9_6616_43ae_9993_903cb25fe08b.slice/crio-cc211167b013afbe3b6a51767d58d743f1208a337258132c000700e7e8ca67f4 WatchSource:0}: Error finding container cc211167b013afbe3b6a51767d58d743f1208a337258132c000700e7e8ca67f4: Status 404 returned error can't find the container with id cc211167b013afbe3b6a51767d58d743f1208a337258132c000700e7e8ca67f4
Sep 10 14:34:34.050455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:34.050419    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx" event={"ID":"f24673a9-6616-43ae-9993-903cb25fe08b","Type":"ContainerStarted","Data":"c00886975f8bd10bc0577469f87b5134b0700b22c81ec25acb398f3b9a6c2498"}
Sep 10 14:34:34.050900 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:34.050490    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:34.050900 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:34.050517    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx" event={"ID":"f24673a9-6616-43ae-9993-903cb25fe08b","Type":"ContainerStarted","Data":"cc211167b013afbe3b6a51767d58d743f1208a337258132c000700e7e8ca67f4"}
Sep 10 14:34:34.069346 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:34.069288    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx" podStartSLOduration=2.069266708 podStartE2EDuration="2.069266708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:34:34.067154111 +0000 UTC m=+879.834876892" watchObservedRunningTime="2026-09-10 14:34:34.069266708 +0000 UTC m=+879.836989493"
Sep 10 14:34:34.724871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:34.724837    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40477f8a-3733-461d-bcba-ec16e2205507" path="/var/lib/kubelet/pods/40477f8a-3733-461d-bcba-ec16e2205507/volumes"
Sep 10 14:34:37.410462 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:37.410423    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:37.511643 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:37.511603    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:42.204264 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:42.204222    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:42.317293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:42.317253    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:42.966717 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:42.966679    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:43.067005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:43.066967    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:45.059414 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.059380    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:34:45.111664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.111631    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"]
Sep 10 14:34:45.111910 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.111885    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb" podUID="7603262d-213b-4732-9f82-238152e59023" containerName="manager" containerID="cri-o://4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a" gracePeriod=10
Sep 10 14:34:45.360008 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.359985    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:45.381843 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.381810    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:45.393685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.393660    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7603262d-213b-4732-9f82-238152e59023-extensions-socket-volume\") pod \"7603262d-213b-4732-9f82-238152e59023\" (UID: \"7603262d-213b-4732-9f82-238152e59023\") "
Sep 10 14:34:45.393841 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.393801    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kstn2\" (UniqueName: \"kubernetes.io/projected/7603262d-213b-4732-9f82-238152e59023-kube-api-access-kstn2\") pod \"7603262d-213b-4732-9f82-238152e59023\" (UID: \"7603262d-213b-4732-9f82-238152e59023\") "
Sep 10 14:34:45.394097 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.394076    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7603262d-213b-4732-9f82-238152e59023-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "7603262d-213b-4732-9f82-238152e59023" (UID: "7603262d-213b-4732-9f82-238152e59023"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:34:45.396047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.396023    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7603262d-213b-4732-9f82-238152e59023-kube-api-access-kstn2" (OuterVolumeSpecName: "kube-api-access-kstn2") pod "7603262d-213b-4732-9f82-238152e59023" (UID: "7603262d-213b-4732-9f82-238152e59023"). InnerVolumeSpecName "kube-api-access-kstn2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:34:45.476894 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.476853    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:45.494496 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.494470    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kstn2\" (UniqueName: \"kubernetes.io/projected/7603262d-213b-4732-9f82-238152e59023-kube-api-access-kstn2\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:45.494496 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:45.494495    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/7603262d-213b-4732-9f82-238152e59023-extensions-socket-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:34:46.111818 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.111782    2578 generic.go:356] "Generic (PLEG): container finished" podID="7603262d-213b-4732-9f82-238152e59023" containerID="4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a" exitCode=0
Sep 10 14:34:46.112211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.111874    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb" event={"ID":"7603262d-213b-4732-9f82-238152e59023","Type":"ContainerDied","Data":"4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a"}
Sep 10 14:34:46.112211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.111887    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"
Sep 10 14:34:46.112211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.111931    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb" event={"ID":"7603262d-213b-4732-9f82-238152e59023","Type":"ContainerDied","Data":"36657374d4c1ad9e1acf28649691f6068d6594ecc0dba07d305849e3a7e6ab14"}
Sep 10 14:34:46.112211 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.111962    2578 scope.go:117] "RemoveContainer" containerID="4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a"
Sep 10 14:34:46.120689 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.120670    2578 scope.go:117] "RemoveContainer" containerID="4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a"
Sep 10 14:34:46.121011 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:34:46.120988    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a\": container with ID starting with 4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a not found: ID does not exist" containerID="4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a"
Sep 10 14:34:46.121114 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.121015    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a"} err="failed to get container status \"4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a\": rpc error: code = NotFound desc = could not find container \"4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a\": container with ID starting with 4d4de6090b97bd23f0a414ba1c742634f63335083b77eda08694fed0ab75c51a not found: ID does not exist"
Sep 10 14:34:46.146005 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.145980    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"]
Sep 10 14:34:46.149705 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.149681    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-q7plb"]
Sep 10 14:34:46.364505 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.364427    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:46.465098 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.465054    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:46.724006 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:46.723920    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7603262d-213b-4732-9f82-238152e59023" path="/var/lib/kubelet/pods/7603262d-213b-4732-9f82-238152e59023/volumes"
Sep 10 14:34:48.471992 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:48.471959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:48.690258 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:48.690228    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:50.963117 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:50.963080    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:51.062158 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:51.062117    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:51.653282 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:51.653246    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:51.754357 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:51.754305    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:51.817130 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:51.817098    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:57.100596 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:57.100562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:34:57.197641 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:34:57.197575    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:01.835620 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:01.835583    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:01.936521 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:01.936486    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:04.021081 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.021044    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:04.025873 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.025846    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6ndfk"]
Sep 10 14:35:04.026184 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.026173    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7603262d-213b-4732-9f82-238152e59023" containerName="manager"
Sep 10 14:35:04.026232 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.026186    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7603262d-213b-4732-9f82-238152e59023" containerName="manager"
Sep 10 14:35:04.026269 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.026242    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7603262d-213b-4732-9f82-238152e59023" containerName="manager"
Sep 10 14:35:04.029192 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.029168    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:04.030984 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.030958    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 10 14:35:04.035681 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.035659    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6ndfk"]
Sep 10 14:35:04.134142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.134107    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6ndfk"]
Sep 10 14:35:04.153692 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.153660    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fc00ba0-7d0e-4533-be17-cfc465b139b1-config-file\") pod \"limitador-limitador-8588749977-6ndfk\" (UID: \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\") " pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:04.153894 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.153769    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k98h5\" (UniqueName: \"kubernetes.io/projected/4fc00ba0-7d0e-4533-be17-cfc465b139b1-kube-api-access-k98h5\") pod \"limitador-limitador-8588749977-6ndfk\" (UID: \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\") " pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:04.201555 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.201516    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:04.254942 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.254913    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k98h5\" (UniqueName: \"kubernetes.io/projected/4fc00ba0-7d0e-4533-be17-cfc465b139b1-kube-api-access-k98h5\") pod \"limitador-limitador-8588749977-6ndfk\" (UID: \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\") " pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:04.255118 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.254984    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fc00ba0-7d0e-4533-be17-cfc465b139b1-config-file\") pod \"limitador-limitador-8588749977-6ndfk\" (UID: \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\") " pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:04.255581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.255562    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fc00ba0-7d0e-4533-be17-cfc465b139b1-config-file\") pod \"limitador-limitador-8588749977-6ndfk\" (UID: \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\") " pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:04.272889 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.272813    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k98h5\" (UniqueName: \"kubernetes.io/projected/4fc00ba0-7d0e-4533-be17-cfc465b139b1-kube-api-access-k98h5\") pod \"limitador-limitador-8588749977-6ndfk\" (UID: \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\") " pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:04.319149 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.319110    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:04.339767 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.339724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:04.611330 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.611170    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6ndfk"]
Sep 10 14:35:04.614353 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:35:04.614322    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fc00ba0_7d0e_4533_be17_cfc465b139b1.slice/crio-b5819dea29976b502ca7701c3b9721d0a0555bf23ba74aa21d26c10b73b53270 WatchSource:0}: Error finding container b5819dea29976b502ca7701c3b9721d0a0555bf23ba74aa21d26c10b73b53270: Status 404 returned error can't find the container with id b5819dea29976b502ca7701c3b9721d0a0555bf23ba74aa21d26c10b73b53270
Sep 10 14:35:04.867609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.867534    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67xgg"]
Sep 10 14:35:04.870309 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.870287    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg"
Sep 10 14:35:04.872024 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.872003    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d85tr\""
Sep 10 14:35:04.876739 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.876715    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67xgg"]
Sep 10 14:35:04.909130 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.909092    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:04.962238 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:04.962211    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzmj2\" (UniqueName: \"kubernetes.io/projected/189e5483-69ac-49d4-914b-7e294b6ef67a-kube-api-access-mzmj2\") pod \"authorino-7bcfd8f9cd-67xgg\" (UID: \"189e5483-69ac-49d4-914b-7e294b6ef67a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg"
Sep 10 14:35:05.006180 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:05.006146    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:05.063155 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:05.063125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzmj2\" (UniqueName: \"kubernetes.io/projected/189e5483-69ac-49d4-914b-7e294b6ef67a-kube-api-access-mzmj2\") pod \"authorino-7bcfd8f9cd-67xgg\" (UID: \"189e5483-69ac-49d4-914b-7e294b6ef67a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg"
Sep 10 14:35:05.072651 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:05.072624    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzmj2\" (UniqueName: \"kubernetes.io/projected/189e5483-69ac-49d4-914b-7e294b6ef67a-kube-api-access-mzmj2\") pod \"authorino-7bcfd8f9cd-67xgg\" (UID: \"189e5483-69ac-49d4-914b-7e294b6ef67a\") " pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg"
Sep 10 14:35:05.182976 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:05.182902    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg"
Sep 10 14:35:05.208330 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:05.208301    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk" event={"ID":"4fc00ba0-7d0e-4533-be17-cfc465b139b1","Type":"ContainerStarted","Data":"b5819dea29976b502ca7701c3b9721d0a0555bf23ba74aa21d26c10b73b53270"}
Sep 10 14:35:05.450601 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:05.450511    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67xgg"]
Sep 10 14:35:05.453344 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:35:05.453311    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod189e5483_69ac_49d4_914b_7e294b6ef67a.slice/crio-b8c25a44cb74ed7bd49ff372f330973dd6f9dafc6452013b34c8586362b368fc WatchSource:0}: Error finding container b8c25a44cb74ed7bd49ff372f330973dd6f9dafc6452013b34c8586362b368fc: Status 404 returned error can't find the container with id b8c25a44cb74ed7bd49ff372f330973dd6f9dafc6452013b34c8586362b368fc
Sep 10 14:35:06.214420 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:06.214325    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg" event={"ID":"189e5483-69ac-49d4-914b-7e294b6ef67a","Type":"ContainerStarted","Data":"b8c25a44cb74ed7bd49ff372f330973dd6f9dafc6452013b34c8586362b368fc"}
Sep 10 14:35:06.215787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:06.215763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk" event={"ID":"4fc00ba0-7d0e-4533-be17-cfc465b139b1","Type":"ContainerStarted","Data":"9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d"}
Sep 10 14:35:06.215897 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:06.215880    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:06.229321 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:06.229269    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk" podStartSLOduration=0.962622506 podStartE2EDuration="2.229249811s" totalImagesPullingTime="1.266627305s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:06.229105096 +0000 UTC m=+911.996827878" watchObservedRunningTime="2026-09-10 14:35:06.229249811 +0000 UTC m=+911.996972596"
Sep 10 14:35:08.214915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:08.214879    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67xgg"]
Sep 10 14:35:08.227546 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:08.227509    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg" event={"ID":"189e5483-69ac-49d4-914b-7e294b6ef67a","Type":"ContainerStarted","Data":"7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab"}
Sep 10 14:35:08.241940 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:08.241888    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg" podStartSLOduration=2.35768391 podStartE2EDuration="4.241871232s" totalImagesPullingTime="1.884187322s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:08.240945396 +0000 UTC m=+914.008668177" watchObservedRunningTime="2026-09-10 14:35:08.241871232 +0000 UTC m=+914.009594013"
Sep 10 14:35:09.231602 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:09.231554    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg" podUID="189e5483-69ac-49d4-914b-7e294b6ef67a" containerName="authorino" containerID="cri-o://7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab" gracePeriod=30
Sep 10 14:35:09.482642 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:09.482588    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg"
Sep 10 14:35:09.600330 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:09.600290    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mzmj2\" (UniqueName: \"kubernetes.io/projected/189e5483-69ac-49d4-914b-7e294b6ef67a-kube-api-access-mzmj2\") pod \"189e5483-69ac-49d4-914b-7e294b6ef67a\" (UID: \"189e5483-69ac-49d4-914b-7e294b6ef67a\") "
Sep 10 14:35:09.602484 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:09.602462    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/189e5483-69ac-49d4-914b-7e294b6ef67a-kube-api-access-mzmj2" (OuterVolumeSpecName: "kube-api-access-mzmj2") pod "189e5483-69ac-49d4-914b-7e294b6ef67a" (UID: "189e5483-69ac-49d4-914b-7e294b6ef67a"). InnerVolumeSpecName "kube-api-access-mzmj2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:09.701585 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:09.701550    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mzmj2\" (UniqueName: \"kubernetes.io/projected/189e5483-69ac-49d4-914b-7e294b6ef67a-kube-api-access-mzmj2\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:35:10.237295 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.237258    2578 generic.go:356] "Generic (PLEG): container finished" podID="189e5483-69ac-49d4-914b-7e294b6ef67a" containerID="7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab" exitCode=0
Sep 10 14:35:10.237738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.237303    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg"
Sep 10 14:35:10.237738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.237310    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg" event={"ID":"189e5483-69ac-49d4-914b-7e294b6ef67a","Type":"ContainerDied","Data":"7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab"}
Sep 10 14:35:10.237738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.237350    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-67xgg" event={"ID":"189e5483-69ac-49d4-914b-7e294b6ef67a","Type":"ContainerDied","Data":"b8c25a44cb74ed7bd49ff372f330973dd6f9dafc6452013b34c8586362b368fc"}
Sep 10 14:35:10.237738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.237371    2578 scope.go:117] "RemoveContainer" containerID="7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab"
Sep 10 14:35:10.246393 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.246371    2578 scope.go:117] "RemoveContainer" containerID="7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab"
Sep 10 14:35:10.246680 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:35:10.246664    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab\": container with ID starting with 7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab not found: ID does not exist" containerID="7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab"
Sep 10 14:35:10.246732 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.246691    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab"} err="failed to get container status \"7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab\": rpc error: code = NotFound desc = could not find container \"7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab\": container with ID starting with 7d3b607530f69c285acaf718e2c1b7bd769508a55ef88d2dd5f98a3a8823bbab not found: ID does not exist"
Sep 10 14:35:10.257251 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.257229    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67xgg"]
Sep 10 14:35:10.261173 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.261150    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-67xgg"]
Sep 10 14:35:10.723423 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:10.723383    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="189e5483-69ac-49d4-914b-7e294b6ef67a" path="/var/lib/kubelet/pods/189e5483-69ac-49d4-914b-7e294b6ef67a/volumes"
Sep 10 14:35:17.222251 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:17.222216    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:19.221715 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.221678    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6ndfk"]
Sep 10 14:35:19.222189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.221915    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk" podUID="4fc00ba0-7d0e-4533-be17-cfc465b139b1" containerName="limitador" containerID="cri-o://9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d" gracePeriod=30
Sep 10 14:35:19.769447 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.769418    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:19.888735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.888697    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k98h5\" (UniqueName: \"kubernetes.io/projected/4fc00ba0-7d0e-4533-be17-cfc465b139b1-kube-api-access-k98h5\") pod \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\" (UID: \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\") "
Sep 10 14:35:19.888735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.888741    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fc00ba0-7d0e-4533-be17-cfc465b139b1-config-file\") pod \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\" (UID: \"4fc00ba0-7d0e-4533-be17-cfc465b139b1\") "
Sep 10 14:35:19.889057 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.889032    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fc00ba0-7d0e-4533-be17-cfc465b139b1-config-file" (OuterVolumeSpecName: "config-file") pod "4fc00ba0-7d0e-4533-be17-cfc465b139b1" (UID: "4fc00ba0-7d0e-4533-be17-cfc465b139b1"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:35:19.891035 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.891014    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fc00ba0-7d0e-4533-be17-cfc465b139b1-kube-api-access-k98h5" (OuterVolumeSpecName: "kube-api-access-k98h5") pod "4fc00ba0-7d0e-4533-be17-cfc465b139b1" (UID: "4fc00ba0-7d0e-4533-be17-cfc465b139b1"). InnerVolumeSpecName "kube-api-access-k98h5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:19.989741 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.989700    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k98h5\" (UniqueName: \"kubernetes.io/projected/4fc00ba0-7d0e-4533-be17-cfc465b139b1-kube-api-access-k98h5\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:35:19.989741 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:19.989736    2578 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4fc00ba0-7d0e-4533-be17-cfc465b139b1-config-file\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:35:20.285597 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.285514    2578 generic.go:356] "Generic (PLEG): container finished" podID="4fc00ba0-7d0e-4533-be17-cfc465b139b1" containerID="9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d" exitCode=0
Sep 10 14:35:20.285597 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.285557    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk"
Sep 10 14:35:20.285597 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.285576    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk" event={"ID":"4fc00ba0-7d0e-4533-be17-cfc465b139b1","Type":"ContainerDied","Data":"9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d"}
Sep 10 14:35:20.286076 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.285618    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-6ndfk" event={"ID":"4fc00ba0-7d0e-4533-be17-cfc465b139b1","Type":"ContainerDied","Data":"b5819dea29976b502ca7701c3b9721d0a0555bf23ba74aa21d26c10b73b53270"}
Sep 10 14:35:20.286076 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.285642    2578 scope.go:117] "RemoveContainer" containerID="9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d"
Sep 10 14:35:20.294213 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.294190    2578 scope.go:117] "RemoveContainer" containerID="9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d"
Sep 10 14:35:20.294454 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:35:20.294435    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d\": container with ID starting with 9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d not found: ID does not exist" containerID="9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d"
Sep 10 14:35:20.294504 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.294462    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d"} err="failed to get container status \"9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d\": rpc error: code = NotFound desc = could not find container \"9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d\": container with ID starting with 9424329f1327430e59f18f686abe2b119d1f11e58dc939cafacbe9ad9192c97d not found: ID does not exist"
Sep 10 14:35:20.305063 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.305013    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6ndfk"]
Sep 10 14:35:20.306534 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.306514    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-6ndfk"]
Sep 10 14:35:20.724293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:20.724256    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fc00ba0-7d0e-4533-be17-cfc465b139b1" path="/var/lib/kubelet/pods/4fc00ba0-7d0e-4533-be17-cfc465b139b1/volumes"
Sep 10 14:35:21.616292 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:21.616258    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:22.010067 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.009980    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"]
Sep 10 14:35:22.010314 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.010301    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="189e5483-69ac-49d4-914b-7e294b6ef67a" containerName="authorino"
Sep 10 14:35:22.010379 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.010315    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="189e5483-69ac-49d4-914b-7e294b6ef67a" containerName="authorino"
Sep 10 14:35:22.010379 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.010334    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fc00ba0-7d0e-4533-be17-cfc465b139b1" containerName="limitador"
Sep 10 14:35:22.010379 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.010339    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fc00ba0-7d0e-4533-be17-cfc465b139b1" containerName="limitador"
Sep 10 14:35:22.010533 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.010405    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fc00ba0-7d0e-4533-be17-cfc465b139b1" containerName="limitador"
Sep 10 14:35:22.010533 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.010415    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="189e5483-69ac-49d4-914b-7e294b6ef67a" containerName="authorino"
Sep 10 14:35:22.013424 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.013404    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:22.015392 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.015363    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 10 14:35:22.015496 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.015424    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 10 14:35:22.015805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.015785    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-jkvns\""
Sep 10 14:35:22.015927 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.015830    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 10 14:35:22.019636 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.019615    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"]
Sep 10 14:35:22.107675 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.107633    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dtsq\" (UniqueName: \"kubernetes.io/projected/b097fd19-98a9-4953-861b-5ebbe643cee4-kube-api-access-8dtsq\") pod \"postgres-69fb4594cf-sxpth\" (UID: \"b097fd19-98a9-4953-861b-5ebbe643cee4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:22.107675 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.107673    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b097fd19-98a9-4953-861b-5ebbe643cee4-data\") pod \"postgres-69fb4594cf-sxpth\" (UID: \"b097fd19-98a9-4953-861b-5ebbe643cee4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:22.208398 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.208362    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dtsq\" (UniqueName: \"kubernetes.io/projected/b097fd19-98a9-4953-861b-5ebbe643cee4-kube-api-access-8dtsq\") pod \"postgres-69fb4594cf-sxpth\" (UID: \"b097fd19-98a9-4953-861b-5ebbe643cee4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:22.208398 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.208396    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b097fd19-98a9-4953-861b-5ebbe643cee4-data\") pod \"postgres-69fb4594cf-sxpth\" (UID: \"b097fd19-98a9-4953-861b-5ebbe643cee4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:22.208739 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.208723    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/b097fd19-98a9-4953-861b-5ebbe643cee4-data\") pod \"postgres-69fb4594cf-sxpth\" (UID: \"b097fd19-98a9-4953-861b-5ebbe643cee4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:22.217850 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.217827    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dtsq\" (UniqueName: \"kubernetes.io/projected/b097fd19-98a9-4953-861b-5ebbe643cee4-kube-api-access-8dtsq\") pod \"postgres-69fb4594cf-sxpth\" (UID: \"b097fd19-98a9-4953-861b-5ebbe643cee4\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:22.325196 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.325170    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:22.597513 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:22.597415    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"]
Sep 10 14:35:22.600185 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:35:22.600161    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb097fd19_98a9_4953_861b_5ebbe643cee4.slice/crio-f21fbfba21a17cf61eecb04323602c6f7e4ca79a4889f69b3cc54e1053c31732 WatchSource:0}: Error finding container f21fbfba21a17cf61eecb04323602c6f7e4ca79a4889f69b3cc54e1053c31732: Status 404 returned error can't find the container with id f21fbfba21a17cf61eecb04323602c6f7e4ca79a4889f69b3cc54e1053c31732
Sep 10 14:35:23.304505 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:23.304456    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth" event={"ID":"b097fd19-98a9-4953-861b-5ebbe643cee4","Type":"ContainerStarted","Data":"f21fbfba21a17cf61eecb04323602c6f7e4ca79a4889f69b3cc54e1053c31732"}
Sep 10 14:35:23.335027 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:23.335000    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:25.317627 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:25.317588    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth" event={"ID":"b097fd19-98a9-4953-861b-5ebbe643cee4","Type":"ContainerStarted","Data":"28f5d3398522a20e51f29e22fce27b0f2fd28c5fa57c0d365df31d1049d66f43"}
Sep 10 14:35:25.318065 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:25.317715    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:25.339899 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:25.339839    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth" podStartSLOduration=1.966356041 podStartE2EDuration="4.339818631s" totalImagesPullingTime="2.37346259s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:25.331296292 +0000 UTC m=+931.099019065" watchObservedRunningTime="2026-09-10 14:35:25.339818631 +0000 UTC m=+931.107541412"
Sep 10 14:35:25.554891 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:25.554839    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:31.351998 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:31.351967    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-sxpth"
Sep 10 14:35:31.635787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:31.635667    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:31.848248 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:31.848209    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:31.972236 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:31.972163    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rgrrc"]
Sep 10 14:35:31.976229 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:31.976207    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:31.978653 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:31.978608    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-d85tr\""
Sep 10 14:35:31.978984 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:31.978916    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 10 14:35:31.982787 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:31.982699    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rgrrc"]
Sep 10 14:35:32.074819 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.074786    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rgrrc"]
Sep 10 14:35:32.075057 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:35:32.075038    2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-x9lsn tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6d4dfc649f-rgrrc" podUID="6f2c826e-8eda-4c33-b8e4-3e9655dca5b5"
Sep 10 14:35:32.095356 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.095320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-tls-cert\") pod \"authorino-6d4dfc649f-rgrrc\" (UID: \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\") " pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:32.095518 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.095468    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x9lsn\" (UniqueName: \"kubernetes.io/projected/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-kube-api-access-x9lsn\") pod \"authorino-6d4dfc649f-rgrrc\" (UID: \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\") " pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:32.196163 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.196125    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x9lsn\" (UniqueName: \"kubernetes.io/projected/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-kube-api-access-x9lsn\") pod \"authorino-6d4dfc649f-rgrrc\" (UID: \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\") " pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:32.196163 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.196169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-tls-cert\") pod \"authorino-6d4dfc649f-rgrrc\" (UID: \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\") " pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:32.198632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.198608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-tls-cert\") pod \"authorino-6d4dfc649f-rgrrc\" (UID: \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\") " pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:32.204011 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.203972    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x9lsn\" (UniqueName: \"kubernetes.io/projected/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-kube-api-access-x9lsn\") pod \"authorino-6d4dfc649f-rgrrc\" (UID: \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\") " pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:32.349615 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.349589    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:32.358833 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.358811    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:32.498283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.498252    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x9lsn\" (UniqueName: \"kubernetes.io/projected/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-kube-api-access-x9lsn\") pod \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\" (UID: \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\") "
Sep 10 14:35:32.498283 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.498291    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-tls-cert\") pod \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\" (UID: \"6f2c826e-8eda-4c33-b8e4-3e9655dca5b5\") "
Sep 10 14:35:32.500441 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.500413    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "6f2c826e-8eda-4c33-b8e4-3e9655dca5b5" (UID: "6f2c826e-8eda-4c33-b8e4-3e9655dca5b5"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:35:32.500548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.500445    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-kube-api-access-x9lsn" (OuterVolumeSpecName: "kube-api-access-x9lsn") pod "6f2c826e-8eda-4c33-b8e4-3e9655dca5b5" (UID: "6f2c826e-8eda-4c33-b8e4-3e9655dca5b5"). InnerVolumeSpecName "kube-api-access-x9lsn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:32.599306 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.599267    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x9lsn\" (UniqueName: \"kubernetes.io/projected/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-kube-api-access-x9lsn\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:35:32.599306 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:32.599299    2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5-tls-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:35:33.354457 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:33.354418    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6d4dfc649f-rgrrc"
Sep 10 14:35:33.378940 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:33.378889    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rgrrc"]
Sep 10 14:35:33.381935 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:33.381908    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6d4dfc649f-rgrrc"]
Sep 10 14:35:34.725087 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:34.725054    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f2c826e-8eda-4c33-b8e4-3e9655dca5b5" path="/var/lib/kubelet/pods/6f2c826e-8eda-4c33-b8e4-3e9655dca5b5/volumes"
Sep 10 14:35:39.082672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.082636    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:39.116856 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.116825    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-56d59ff8b7-vfcmd"]
Sep 10 14:35:39.125335 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.125311    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.127293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.127268    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 14:35:39.127293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.127281    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 14:35:39.127293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.127302    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 14:35:39.127551 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.127276    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-754rh\""
Sep 10 14:35:39.132061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.132017    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56d59ff8b7-vfcmd"]
Sep 10 14:35:39.151234 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.151206    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.151234 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.151238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9482\" (UniqueName: \"kubernetes.io/projected/01e1e45c-c46e-4e37-ae25-49cea5ade75d-kube-api-access-d9482\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.151428 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.151258    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-webhook-cert\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.157649 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.157617    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:39.215051 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.215014    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:39.252069 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.252034    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.252069 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.252070    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9482\" (UniqueName: \"kubernetes.io/projected/01e1e45c-c46e-4e37-ae25-49cea5ade75d-kube-api-access-d9482\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.252278 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.252089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-webhook-cert\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.252278 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:35:39.252203    2578 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 14:35:39.252351 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:35:39.252288    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs podName:01e1e45c-c46e-4e37-ae25-49cea5ade75d nodeName:}" failed. No retries permitted until 2026-09-10 14:35:39.75226778 +0000 UTC m=+945.519990546 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs") pod "maas-controller-56d59ff8b7-vfcmd" (UID: "01e1e45c-c46e-4e37-ae25-49cea5ade75d") : secret "maas-controller-metrics-tls" not found
Sep 10 14:35:39.254638 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.254621    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-webhook-cert\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.266149 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.266105    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9482\" (UniqueName: \"kubernetes.io/projected/01e1e45c-c46e-4e37-ae25-49cea5ade75d-kube-api-access-d9482\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.281578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.281550    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55b7c7d477-rzw7w"]
Sep 10 14:35:39.285076 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.285057    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.287257 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.287231    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:39.295276 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.295253    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-rzw7w"]
Sep 10 14:35:39.353012 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.352927    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-webhook-cert\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.353012 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.352976    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.353205 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.353019    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmx2k\" (UniqueName: \"kubernetes.io/projected/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-kube-api-access-bmx2k\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.454241 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.454203    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmx2k\" (UniqueName: \"kubernetes.io/projected/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-kube-api-access-bmx2k\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.454415 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.454260    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-webhook-cert\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.454415 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.454293    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.454415 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:35:39.454383    2578 secret.go:189] Couldn't get secret opendatahub/maas-controller-metrics-tls: secret "maas-controller-metrics-tls" not found
Sep 10 14:35:39.454524 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:35:39.454441    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs podName:a7e52a2b-6e5e-4f5f-bc86-904520f300ee nodeName:}" failed. No retries permitted until 2026-09-10 14:35:39.954422725 +0000 UTC m=+945.722145504 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs") pod "maas-controller-55b7c7d477-rzw7w" (UID: "a7e52a2b-6e5e-4f5f-bc86-904520f300ee") : secret "maas-controller-metrics-tls" not found
Sep 10 14:35:39.456909 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.456883    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-webhook-cert\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.462290 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.462267    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmx2k\" (UniqueName: \"kubernetes.io/projected/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-kube-api-access-bmx2k\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.501672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.501640    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:39.757176 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.757068    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.759722 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.759689    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs\") pod \"maas-controller-56d59ff8b7-vfcmd\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") " pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:39.958269 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.958229    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:39.960803 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:39.960771    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs\") pod \"maas-controller-55b7c7d477-rzw7w\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") " pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:40.037651 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:40.037557    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:40.196530 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:40.196492    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:40.308360 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:40.308334    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-56d59ff8b7-vfcmd"]
Sep 10 14:35:40.310464 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:35:40.310440    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01e1e45c_c46e_4e37_ae25_49cea5ade75d.slice/crio-6368f21c1fcaa6a128c74ec356a7e379545f4983b408a0980286ad5ef78ef13b WatchSource:0}: Error finding container 6368f21c1fcaa6a128c74ec356a7e379545f4983b408a0980286ad5ef78ef13b: Status 404 returned error can't find the container with id 6368f21c1fcaa6a128c74ec356a7e379545f4983b408a0980286ad5ef78ef13b
Sep 10 14:35:40.390579 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:40.390536    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd" event={"ID":"01e1e45c-c46e-4e37-ae25-49cea5ade75d","Type":"ContainerStarted","Data":"6368f21c1fcaa6a128c74ec356a7e379545f4983b408a0980286ad5ef78ef13b"}
Sep 10 14:35:40.470937 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:40.470859    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-rzw7w"]
Sep 10 14:35:40.473181 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:35:40.473158    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7e52a2b_6e5e_4f5f_bc86_904520f300ee.slice/crio-1e3ef7a8d07d2bacd13616f219db04130bce78a7e7021ba6b76992c17647069d WatchSource:0}: Error finding container 1e3ef7a8d07d2bacd13616f219db04130bce78a7e7021ba6b76992c17647069d: Status 404 returned error can't find the container with id 1e3ef7a8d07d2bacd13616f219db04130bce78a7e7021ba6b76992c17647069d
Sep 10 14:35:41.402868 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:41.402774    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-rzw7w" event={"ID":"a7e52a2b-6e5e-4f5f-bc86-904520f300ee","Type":"ContainerStarted","Data":"1e3ef7a8d07d2bacd13616f219db04130bce78a7e7021ba6b76992c17647069d"}
Sep 10 14:35:43.164144 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:43.164042    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:43.311165 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:43.311104    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:43.415619 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:43.415511    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-rzw7w" event={"ID":"a7e52a2b-6e5e-4f5f-bc86-904520f300ee","Type":"ContainerStarted","Data":"fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516"}
Sep 10 14:35:43.415824 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:43.415617    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:43.417249 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:43.417215    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd" event={"ID":"01e1e45c-c46e-4e37-ae25-49cea5ade75d","Type":"ContainerStarted","Data":"70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9"}
Sep 10 14:35:43.417394 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:43.417350    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:43.438539 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:43.438480    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55b7c7d477-rzw7w" podStartSLOduration=1.974173795 podStartE2EDuration="4.438465145s" totalImagesPullingTime="2.46429135s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:43.436206689 +0000 UTC m=+949.203929468" watchObservedRunningTime="2026-09-10 14:35:43.438465145 +0000 UTC m=+949.206188245"
Sep 10 14:35:43.454428 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:43.454380    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd" podStartSLOduration=1.839345002 podStartE2EDuration="4.454365055s" totalImagesPullingTime="2.615020053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:35:43.452007522 +0000 UTC m=+949.219730302" watchObservedRunningTime="2026-09-10 14:35:43.454365055 +0000 UTC m=+949.222087834"
Sep 10 14:35:54.424471 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.424439    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:54.424886 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.424514    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:35:54.488532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.488481    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56d59ff8b7-vfcmd"]
Sep 10 14:35:54.488900 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.488847    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd" podUID="01e1e45c-c46e-4e37-ae25-49cea5ade75d" containerName="manager" containerID="cri-o://70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9" gracePeriod=30
Sep 10 14:35:54.743441 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.743411    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:54.815630 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.815596    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:54.868785 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.868731    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-webhook-cert\") pod \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") "
Sep 10 14:35:54.868978 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.868800    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs\") pod \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") "
Sep 10 14:35:54.868978 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.868859    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d9482\" (UniqueName: \"kubernetes.io/projected/01e1e45c-c46e-4e37-ae25-49cea5ade75d-kube-api-access-d9482\") pod \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\" (UID: \"01e1e45c-c46e-4e37-ae25-49cea5ade75d\") "
Sep 10 14:35:54.871471 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.871432    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:35:54.871646 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.871611    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "01e1e45c-c46e-4e37-ae25-49cea5ade75d" (UID: "01e1e45c-c46e-4e37-ae25-49cea5ade75d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:35:54.871646 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.871622    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01e1e45c-c46e-4e37-ae25-49cea5ade75d-kube-api-access-d9482" (OuterVolumeSpecName: "kube-api-access-d9482") pod "01e1e45c-c46e-4e37-ae25-49cea5ade75d" (UID: "01e1e45c-c46e-4e37-ae25-49cea5ade75d"). InnerVolumeSpecName "kube-api-access-d9482". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:35:54.871646 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.871640    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "01e1e45c-c46e-4e37-ae25-49cea5ade75d" (UID: "01e1e45c-c46e-4e37-ae25-49cea5ade75d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:35:54.969969 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.969881    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d9482\" (UniqueName: \"kubernetes.io/projected/01e1e45c-c46e-4e37-ae25-49cea5ade75d-kube-api-access-d9482\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:35:54.969969 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.969909    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-webhook-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:35:54.969969 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:54.969920    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01e1e45c-c46e-4e37-ae25-49cea5ade75d-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:35:55.484810 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.484767    2578 generic.go:356] "Generic (PLEG): container finished" podID="01e1e45c-c46e-4e37-ae25-49cea5ade75d" containerID="70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9" exitCode=0
Sep 10 14:35:55.485231 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.484823    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd"
Sep 10 14:35:55.485231 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.484865    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd" event={"ID":"01e1e45c-c46e-4e37-ae25-49cea5ade75d","Type":"ContainerDied","Data":"70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9"}
Sep 10 14:35:55.485231 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.484913    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-56d59ff8b7-vfcmd" event={"ID":"01e1e45c-c46e-4e37-ae25-49cea5ade75d","Type":"ContainerDied","Data":"6368f21c1fcaa6a128c74ec356a7e379545f4983b408a0980286ad5ef78ef13b"}
Sep 10 14:35:55.485231 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.484934    2578 scope.go:117] "RemoveContainer" containerID="70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9"
Sep 10 14:35:55.493801 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.493620    2578 scope.go:117] "RemoveContainer" containerID="70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9"
Sep 10 14:35:55.493967 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:35:55.493950    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9\": container with ID starting with 70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9 not found: ID does not exist" containerID="70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9"
Sep 10 14:35:55.494011 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.493977    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9"} err="failed to get container status \"70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9\": rpc error: code = NotFound desc = could not find container \"70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9\": container with ID starting with 70a9fba654ca1c0e3d9cda8caac67673767cb3a4ec86515428d37c63872879e9 not found: ID does not exist"
Sep 10 14:35:55.505040 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.505005    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-56d59ff8b7-vfcmd"]
Sep 10 14:35:55.509048 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:55.509017    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-56d59ff8b7-vfcmd"]
Sep 10 14:35:56.724387 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:35:56.724354    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="01e1e45c-c46e-4e37-ae25-49cea5ade75d" path="/var/lib/kubelet/pods/01e1e45c-c46e-4e37-ae25-49cea5ade75d/volumes"
Sep 10 14:36:08.500838 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:08.500788    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:08.555534 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:08.555496    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:12.587219 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:12.587167    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:12.800082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:12.800045    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.143730 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.143697    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.243821 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.243770    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.368146 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.368113    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"]
Sep 10 14:36:14.368497 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.368484    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="01e1e45c-c46e-4e37-ae25-49cea5ade75d" containerName="manager"
Sep 10 14:36:14.368538 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.368499    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="01e1e45c-c46e-4e37-ae25-49cea5ade75d" containerName="manager"
Sep 10 14:36:14.368573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.368551    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="01e1e45c-c46e-4e37-ae25-49cea5ade75d" containerName="manager"
Sep 10 14:36:14.372837 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.372816    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.374552 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.374521    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 10 14:36:14.374719 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.374703    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 10 14:36:14.374802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.374787    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9r2v9\""
Sep 10 14:36:14.382981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.382955    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"]
Sep 10 14:36:14.435596 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.435498    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-maas-api-tls\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.435596 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.435547    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.435864 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.435602    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v5t5\" (UniqueName: \"kubernetes.io/projected/e09018de-91a7-4ed3-b108-10af74121521-kube-api-access-9v5t5\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.435864 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.435623    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e09018de-91a7-4ed3-b108-10af74121521-openshift-service-ca\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.528328 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.528293    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.536071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.536048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e09018de-91a7-4ed3-b108-10af74121521-openshift-service-ca\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.536224 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.536118    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-maas-api-tls\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.536224 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.536152    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.536224 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.536190    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v5t5\" (UniqueName: \"kubernetes.io/projected/e09018de-91a7-4ed3-b108-10af74121521-kube-api-access-9v5t5\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.536368 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:36:14.536232    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 10 14:36:14.536368 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:36:14.536314    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs podName:e09018de-91a7-4ed3-b108-10af74121521 nodeName:}" failed. No retries permitted until 2026-09-10 14:36:15.036292161 +0000 UTC m=+980.804014933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs") pod "maas-api-6dd849cfd5-b6mpl" (UID: "e09018de-91a7-4ed3-b108-10af74121521") : secret "maas-api-metrics-tls" not found
Sep 10 14:36:14.536802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.536772    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e09018de-91a7-4ed3-b108-10af74121521-openshift-service-ca\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.538887 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.538864    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-maas-api-tls\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.546560 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.546535    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v5t5\" (UniqueName: \"kubernetes.io/projected/e09018de-91a7-4ed3-b108-10af74121521-kube-api-access-9v5t5\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:14.642853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.642816    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.764379 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.764295    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.869818 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.869778    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:14.893833 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.893795    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"]
Sep 10 14:36:14.898092 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.898067    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:14.899946 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.899925    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-5bz5k\""
Sep 10 14:36:14.900090 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.899934    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 10 14:36:14.907146 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:14.907118    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"]
Sep 10 14:36:15.041260 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.041165    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4ce8ee1-18ff-495f-a1d6-20320ed2118d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-vjxxl\" (UID: \"c4ce8ee1-18ff-495f-a1d6-20320ed2118d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:15.041260 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.041222    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:15.041498 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.041345    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sv9mb\" (UniqueName: \"kubernetes.io/projected/c4ce8ee1-18ff-495f-a1d6-20320ed2118d-kube-api-access-sv9mb\") pod \"payload-processing-5b54b7fbb7-vjxxl\" (UID: \"c4ce8ee1-18ff-495f-a1d6-20320ed2118d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:15.044065 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.044042    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs\") pod \"maas-api-6dd849cfd5-b6mpl\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") " pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:15.141981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.141938    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sv9mb\" (UniqueName: \"kubernetes.io/projected/c4ce8ee1-18ff-495f-a1d6-20320ed2118d-kube-api-access-sv9mb\") pod \"payload-processing-5b54b7fbb7-vjxxl\" (UID: \"c4ce8ee1-18ff-495f-a1d6-20320ed2118d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:15.142161 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.142048    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4ce8ee1-18ff-495f-a1d6-20320ed2118d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-vjxxl\" (UID: \"c4ce8ee1-18ff-495f-a1d6-20320ed2118d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:15.142913 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.142888    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c4ce8ee1-18ff-495f-a1d6-20320ed2118d-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-vjxxl\" (UID: \"c4ce8ee1-18ff-495f-a1d6-20320ed2118d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:15.149382 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.149360    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sv9mb\" (UniqueName: \"kubernetes.io/projected/c4ce8ee1-18ff-495f-a1d6-20320ed2118d-kube-api-access-sv9mb\") pod \"payload-processing-5b54b7fbb7-vjxxl\" (UID: \"c4ce8ee1-18ff-495f-a1d6-20320ed2118d\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:15.210768 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.210724    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:15.284450 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.284422    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:15.484704 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.484670    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"]
Sep 10 14:36:15.487843 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:36:15.487807    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4ce8ee1_18ff_495f_a1d6_20320ed2118d.slice/crio-057eef7ef0ad08f43ebd3996ebded2bd3f7ea33818c4482c6f78285f14f4dc56 WatchSource:0}: Error finding container 057eef7ef0ad08f43ebd3996ebded2bd3f7ea33818c4482c6f78285f14f4dc56: Status 404 returned error can't find the container with id 057eef7ef0ad08f43ebd3996ebded2bd3f7ea33818c4482c6f78285f14f4dc56
Sep 10 14:36:15.560966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.560858    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"]
Sep 10 14:36:15.564624 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:36:15.564593    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode09018de_91a7_4ed3_b108_10af74121521.slice/crio-f608f8f727a4f393afa7da8fe00687e94f2bf7d4ef935ecbbe5a929f41d9bc68 WatchSource:0}: Error finding container f608f8f727a4f393afa7da8fe00687e94f2bf7d4ef935ecbbe5a929f41d9bc68: Status 404 returned error can't find the container with id f608f8f727a4f393afa7da8fe00687e94f2bf7d4ef935ecbbe5a929f41d9bc68
Sep 10 14:36:15.586065 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.586034    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl" event={"ID":"e09018de-91a7-4ed3-b108-10af74121521","Type":"ContainerStarted","Data":"f608f8f727a4f393afa7da8fe00687e94f2bf7d4ef935ecbbe5a929f41d9bc68"}
Sep 10 14:36:15.587672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:15.587633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl" event={"ID":"c4ce8ee1-18ff-495f-a1d6-20320ed2118d","Type":"ContainerStarted","Data":"057eef7ef0ad08f43ebd3996ebded2bd3f7ea33818c4482c6f78285f14f4dc56"}
Sep 10 14:36:18.607049 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:18.606993    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl" event={"ID":"c4ce8ee1-18ff-495f-a1d6-20320ed2118d","Type":"ContainerStarted","Data":"d626d72164b34a9a412d75161fba00c500b2afcf7804f27b350030d697a56797"}
Sep 10 14:36:18.607515 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:18.607065    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:18.608598 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:18.608572    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl" event={"ID":"e09018de-91a7-4ed3-b108-10af74121521","Type":"ContainerStarted","Data":"2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594"}
Sep 10 14:36:18.608685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:18.608623    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:18.626227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:18.626171    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl" podStartSLOduration=2.548189701 podStartE2EDuration="4.626152785s" totalImagesPullingTime="2.077963084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:18.624190493 +0000 UTC m=+984.391913273" watchObservedRunningTime="2026-09-10 14:36:18.626152785 +0000 UTC m=+984.393875565"
Sep 10 14:36:18.641097 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:18.641042    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl" podStartSLOduration=2.6452830499999997 podStartE2EDuration="4.641019181s" totalImagesPullingTime="1.995736131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:18.639023581 +0000 UTC m=+984.406746356" watchObservedRunningTime="2026-09-10 14:36:18.641019181 +0000 UTC m=+984.408741963"
Sep 10 14:36:24.619322 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:24.619292    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 14:36:25.592786 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:25.592733    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:25.691251 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:25.691207    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:29.616215 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:29.616179    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-vjxxl"
Sep 10 14:36:31.472319 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:31.472280    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:37.133442 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.133408    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"]
Sep 10 14:36:37.137293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.137266    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.139459 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.139428    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 10 14:36:37.140040 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.140017    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 10 14:36:37.140179 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.140020    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-n6q9c\""
Sep 10 14:36:37.140179 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.140021    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 10 14:36:37.148145 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.148112    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"]
Sep 10 14:36:37.149623 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.149595    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.149720 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.149689    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6fq7\" (UniqueName: \"kubernetes.io/projected/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-kube-api-access-f6fq7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.149809 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.149722    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.149809 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.149780    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.149904 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.149809    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.149904 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.149848    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.169999 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.169952    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:37.250297 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250250    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6fq7\" (UniqueName: \"kubernetes.io/projected/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-kube-api-access-f6fq7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.250297 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250302    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.250574 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250331    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.250574 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250356    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.250574 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250383    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.250574 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250405    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.250855 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250834    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.250906 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250845    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.250906 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.250886    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.252990 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.252959    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.253164 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.253147    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.259288 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.259259    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6fq7\" (UniqueName: \"kubernetes.io/projected/d8453cc8-efd1-44ac-93b6-0a1dea27db7c-kube-api-access-f6fq7\") pod \"e2e-distinct-simulated-kserve-59975d69f9-gf4qb\" (UID: \"d8453cc8-efd1-44ac-93b6-0a1dea27db7c\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.449509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.449411    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:37.728865 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.728781    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"]
Sep 10 14:36:37.731807 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:36:37.731776    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8453cc8_efd1_44ac_93b6_0a1dea27db7c.slice/crio-919aa7c67de96a6d64b1f99db65c87237b411287d24d8a4ced3a28769196803c WatchSource:0}: Error finding container 919aa7c67de96a6d64b1f99db65c87237b411287d24d8a4ced3a28769196803c: Status 404 returned error can't find the container with id 919aa7c67de96a6d64b1f99db65c87237b411287d24d8a4ced3a28769196803c
Sep 10 14:36:37.732970 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:37.732951    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:36:38.714282 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:38.714244    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb" event={"ID":"d8453cc8-efd1-44ac-93b6-0a1dea27db7c","Type":"ContainerStarted","Data":"919aa7c67de96a6d64b1f99db65c87237b411287d24d8a4ced3a28769196803c"}
Sep 10 14:36:39.721190 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:39.721154    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb" event={"ID":"d8453cc8-efd1-44ac-93b6-0a1dea27db7c","Type":"ContainerStarted","Data":"b7c092ef43091300e6e3f5afef72f769e3e83a56f861e5cbb6d76904ed98e7ff"}
Sep 10 14:36:39.741790 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:39.741715    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb" podStartSLOduration=1.717763502 podStartE2EDuration="2.741698754s" totalImagesPullingTime="1.023935252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:36:39.740003382 +0000 UTC m=+1005.507726163" watchObservedRunningTime="2026-09-10 14:36:39.741698754 +0000 UTC m=+1005.509421538"
Sep 10 14:36:46.365355 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:46.365301    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:36:47.450171 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:47.450127    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:47.450613 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:47.450197    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:47.463281 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:47.463244    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:47.775637 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:47.775563    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-gf4qb"
Sep 10 14:36:57.084710 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:36:57.084671    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:02.931595 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:02.931553    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"]
Sep 10 14:37:02.938288 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:02.938258    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:02.940377 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:02.940351    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 10 14:37:02.944325 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:02.944299    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"]
Sep 10 14:37:02.969035 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:02.969002    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:03.079823 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.079783    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.080029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.079850    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.080029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.079874    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9wtk\" (UniqueName: \"kubernetes.io/projected/4546e28e-4ef4-425a-aaa8-8288b34e99f4-kube-api-access-v9wtk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.080029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.079907    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.080029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.079959    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4546e28e-4ef4-425a-aaa8-8288b34e99f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.080029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.079995    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.180894 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.180855    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.180894 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.180892    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9wtk\" (UniqueName: \"kubernetes.io/projected/4546e28e-4ef4-425a-aaa8-8288b34e99f4-kube-api-access-v9wtk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.181149 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.180919    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.181149 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.180967    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4546e28e-4ef4-425a-aaa8-8288b34e99f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.181149 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.181009    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.181149 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.181067    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.181384 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.181282    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.181518 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.181481    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.181518 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.181490    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.183380 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.183338    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4546e28e-4ef4-425a-aaa8-8288b34e99f4-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.183734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.183713    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4546e28e-4ef4-425a-aaa8-8288b34e99f4-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.188204 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.188184    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9wtk\" (UniqueName: \"kubernetes.io/projected/4546e28e-4ef4-425a-aaa8-8288b34e99f4-kube-api-access-v9wtk\") pod \"e2e-trlp-test-simulated-kserve-598466548d-wlzxh\" (UID: \"4546e28e-4ef4-425a-aaa8-8288b34e99f4\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.249902 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.249856    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:03.528992 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.528962    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"]
Sep 10 14:37:03.530611 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:37:03.530576    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4546e28e_4ef4_425a_aaa8_8288b34e99f4.slice/crio-6f2e89848ca60a5cce24194406800e358b55fad05c82965f9d87a6e1e63b1ab1 WatchSource:0}: Error finding container 6f2e89848ca60a5cce24194406800e358b55fad05c82965f9d87a6e1e63b1ab1: Status 404 returned error can't find the container with id 6f2e89848ca60a5cce24194406800e358b55fad05c82965f9d87a6e1e63b1ab1
Sep 10 14:37:03.845066 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.845030    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh" event={"ID":"4546e28e-4ef4-425a-aaa8-8288b34e99f4","Type":"ContainerStarted","Data":"13f3ccc210a3dad090f9fe0f586ea00a0bb9ce39e9cd879881fc16a61075c330"}
Sep 10 14:37:03.845252 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.845086    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh" event={"ID":"4546e28e-4ef4-425a-aaa8-8288b34e99f4","Type":"ContainerStarted","Data":"6f2e89848ca60a5cce24194406800e358b55fad05c82965f9d87a6e1e63b1ab1"}
Sep 10 14:37:03.864103 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:03.864031    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh" podStartSLOduration=1.666142681 podStartE2EDuration="1.864014742s" totalImagesPullingTime="197.872061ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:03.863111338 +0000 UTC m=+1029.630834119" watchObservedRunningTime="2026-09-10 14:37:03.864014742 +0000 UTC m=+1029.631737557"
Sep 10 14:37:13.250664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:13.250603    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:13.250664 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:13.250684    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:13.263335 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:13.263300    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:13.906415 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:13.906383    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-wlzxh"
Sep 10 14:37:26.477466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:26.477420    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:35.728603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.728569    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"]
Sep 10 14:37:35.732268 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.732246    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.733958 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.733941    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 10 14:37:35.742205 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.742179    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"]
Sep 10 14:37:35.783860 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.783823    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.784047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.783881    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.784047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.783920    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/184501d0-aeb8-4313-8358-50c8d1ddb772-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.784047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.783988    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.784047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.784027    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.784224 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.784057    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6j885\" (UniqueName: \"kubernetes.io/projected/184501d0-aeb8-4313-8358-50c8d1ddb772-kube-api-access-6j885\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.823677 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.823638    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:35.885207 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885163    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.885397 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885219    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.885397 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885259    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6j885\" (UniqueName: \"kubernetes.io/projected/184501d0-aeb8-4313-8358-50c8d1ddb772-kube-api-access-6j885\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.885516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885401    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.885516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885473    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.885516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885507    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/184501d0-aeb8-4313-8358-50c8d1ddb772-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.885679 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885569    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.885679 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885630    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.885911 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.885884    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.887963 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.887941    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/184501d0-aeb8-4313-8358-50c8d1ddb772-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.888121 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.888102    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/184501d0-aeb8-4313-8358-50c8d1ddb772-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:35.899219 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:35.899182    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6j885\" (UniqueName: \"kubernetes.io/projected/184501d0-aeb8-4313-8358-50c8d1ddb772-kube-api-access-6j885\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-s2fg2\" (UID: \"184501d0-aeb8-4313-8358-50c8d1ddb772\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:36.044199 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:36.044108    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:36.320707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:36.320680    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"]
Sep 10 14:37:36.323002 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:37:36.322961    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod184501d0_aeb8_4313_8358_50c8d1ddb772.slice/crio-0b68437f3b50f294639316a52eddf1622082bd68770674697dc92b6ffd950ba0 WatchSource:0}: Error finding container 0b68437f3b50f294639316a52eddf1622082bd68770674697dc92b6ffd950ba0: Status 404 returned error can't find the container with id 0b68437f3b50f294639316a52eddf1622082bd68770674697dc92b6ffd950ba0
Sep 10 14:37:37.011615 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:37.011530    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2" event={"ID":"184501d0-aeb8-4313-8358-50c8d1ddb772","Type":"ContainerStarted","Data":"61321cf0596713046b3fd884b64c0c17210096edb1311c1f68522b6058ad645e"}
Sep 10 14:37:37.011615 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:37.011586    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2" event={"ID":"184501d0-aeb8-4313-8358-50c8d1ddb772","Type":"ContainerStarted","Data":"0b68437f3b50f294639316a52eddf1622082bd68770674697dc92b6ffd950ba0"}
Sep 10 14:37:37.031580 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:37.031523    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2" podStartSLOduration=1.815237796 podStartE2EDuration="2.031506489s" totalImagesPullingTime="216.268693ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:37:37.02937369 +0000 UTC m=+1062.797096470" watchObservedRunningTime="2026-09-10 14:37:37.031506489 +0000 UTC m=+1062.799229320"
Sep 10 14:37:42.346361 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:42.346322    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:37:46.044998 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:46.044908    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:46.045445 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:46.045015    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:46.057631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:46.057606    2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:37:46.069354 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:37:46.069327    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-s2fg2"
Sep 10 14:38:31.741252 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:31.741207    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:31.842131 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:31.842080    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:32.813669 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:32.813610    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:32.916713 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:32.916680    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:48.089306 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:48.089244    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:48.223679 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:48.223644    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:48.404109 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:48.404020    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:48.502021 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:48.501977    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:49.468168 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:49.468130    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:49.559522 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:49.559493    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:50.018222 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:50.018192    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:51.362185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:51.362149    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:51.447544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:51.447508    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:52.633076 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.633041    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"]
Sep 10 14:38:52.637241 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.637214    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.639307 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.639282    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-c892fe-serving-cert\""
Sep 10 14:38:52.648432 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.648407    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"]
Sep 10 14:38:52.679426 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.679388    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.679606 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.679451    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-metrics-certs\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.679606 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.679492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mf4jp\" (UniqueName: \"kubernetes.io/projected/1c801c14-8727-4b1a-9de4-757b247b7c21-kube-api-access-mf4jp\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.679738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.679601    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c801c14-8727-4b1a-9de4-757b247b7c21-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.681222 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.681193    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:52.774566 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.774526    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:52.780601 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.780566    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c801c14-8727-4b1a-9de4-757b247b7c21-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.780805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.780694    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.780805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.780776    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-metrics-certs\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.780805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.780795    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mf4jp\" (UniqueName: \"kubernetes.io/projected/1c801c14-8727-4b1a-9de4-757b247b7c21-kube-api-access-mf4jp\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.780986 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:38:52.780883    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-serving-cert: secret "maas-api-e2e-inf-a-c892fe-serving-cert" not found
Sep 10 14:38:52.780986 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:38:52.780971    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls podName:1c801c14-8727-4b1a-9de4-757b247b7c21 nodeName:}" failed. No retries permitted until 2026-09-10 14:38:53.280948733 +0000 UTC m=+1139.048671493 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls") pod "maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t" (UID: "1c801c14-8727-4b1a-9de4-757b247b7c21") : secret "maas-api-e2e-inf-a-c892fe-serving-cert" not found
Sep 10 14:38:52.781889 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.781861    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c801c14-8727-4b1a-9de4-757b247b7c21-openshift-service-ca\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.783734 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.783710    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-metrics-certs\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:52.792388 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:52.792355    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mf4jp\" (UniqueName: \"kubernetes.io/projected/1c801c14-8727-4b1a-9de4-757b247b7c21-kube-api-access-mf4jp\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:53.155076 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:53.155044    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:53.255970 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:53.255912    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:53.285103 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:53.285069    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:53.287774 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:53.287728    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls\") pod \"maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:53.431740 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:53.431649    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:53.549795 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:53.549737    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:53.825033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:53.825007    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"]
Sep 10 14:38:53.827822 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:38:53.827789    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1c801c14_8727_4b1a_9de4_757b247b7c21.slice/crio-d2502471ace3443aa4ded11524b33c1cd87d6f31b3df39ec07057b6166ae241d WatchSource:0}: Error finding container d2502471ace3443aa4ded11524b33c1cd87d6f31b3df39ec07057b6166ae241d: Status 404 returned error can't find the container with id d2502471ace3443aa4ded11524b33c1cd87d6f31b3df39ec07057b6166ae241d
Sep 10 14:38:54.394097 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:54.394054    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t" event={"ID":"1c801c14-8727-4b1a-9de4-757b247b7c21","Type":"ContainerStarted","Data":"eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87"}
Sep 10 14:38:54.394263 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:54.394223    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:38:54.394263 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:54.394245    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t" event={"ID":"1c801c14-8727-4b1a-9de4-757b247b7c21","Type":"ContainerStarted","Data":"d2502471ace3443aa4ded11524b33c1cd87d6f31b3df39ec07057b6166ae241d"}
Sep 10 14:38:54.410086 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:54.410035    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t" podStartSLOduration=1.9506837030000002 podStartE2EDuration="2.410018223s" totalImagesPullingTime="459.33452ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:38:54.409242181 +0000 UTC m=+1140.176964963" watchObservedRunningTime="2026-09-10 14:38:54.410018223 +0000 UTC m=+1140.177741374"
Sep 10 14:38:55.401478 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:55.401451    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:55.569918 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:55.569880    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:55.640209 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:55.640172    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:55.716173 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:55.716087    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:55.780177 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:55.780139    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:56.084564 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:56.084530    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:56.182314 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:56.182277    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:56.314213 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:56.314185    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:58.322646 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:58.322613    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:58.415759 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:58.415658    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:58.781094 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:58.781011    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:58.914402 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:58.914372    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:59.537308 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.537277    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"]
Sep 10 14:38:59.541093 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.541074    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.542906 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.542888    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-e7415a-serving-cert\""
Sep 10 14:38:59.550142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.550119    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"]
Sep 10 14:38:59.625789 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.625731    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:59.648145 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.648113    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntq5f\" (UniqueName: \"kubernetes.io/projected/64a334e0-45af-47e8-b6ca-33c1713d72e9-kube-api-access-ntq5f\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.648145 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.648148    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64a334e0-45af-47e8-b6ca-33c1713d72e9-openshift-service-ca\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.648339 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.648170    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.648339 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.648249    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-metrics-certs\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.727203 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.727166    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:38:59.749438 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.749407    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntq5f\" (UniqueName: \"kubernetes.io/projected/64a334e0-45af-47e8-b6ca-33c1713d72e9-kube-api-access-ntq5f\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.749561 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.749459    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64a334e0-45af-47e8-b6ca-33c1713d72e9-openshift-service-ca\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.749561 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.749499    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.749645 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.749569    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-metrics-certs\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.749695 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:38:59.749678    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-serving-cert: secret "maas-api-e2e-autores-e7415a-serving-cert" not found
Sep 10 14:38:59.749794 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:38:59.749778    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls podName:64a334e0-45af-47e8-b6ca-33c1713d72e9 nodeName:}" failed. No retries permitted until 2026-09-10 14:39:00.249731553 +0000 UTC m=+1146.017454317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls") pod "maas-api-e2e-autores-e7415a-557d475db6-7htsp" (UID: "64a334e0-45af-47e8-b6ca-33c1713d72e9") : secret "maas-api-e2e-autores-e7415a-serving-cert" not found
Sep 10 14:38:59.750267 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.750242    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64a334e0-45af-47e8-b6ca-33c1713d72e9-openshift-service-ca\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.752246 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.752224    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-metrics-certs\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.757104 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.757086    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntq5f\" (UniqueName: \"kubernetes.io/projected/64a334e0-45af-47e8-b6ca-33c1713d72e9-kube-api-access-ntq5f\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:38:59.867693 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:38:59.867644    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:00.200471 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.200384    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:00.253182 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.253150    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:39:00.255716 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.255694    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls\") pod \"maas-api-e2e-autores-e7415a-557d475db6-7htsp\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:39:00.300834 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.300803    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"]
Sep 10 14:39:00.304524 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.304509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:00.306798 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.306735    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-e7415a-dockercfg-tsg8v\""
Sep 10 14:39:00.306965 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.306944    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-e7415a\""
Sep 10 14:39:00.314201 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.314176    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"]
Sep 10 14:39:00.353859 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.353828    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7vtw2\" (UniqueName: \"kubernetes.io/projected/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-kube-api-access-7vtw2\") pod \"payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w\" (UID: \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:00.354003 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.353888    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w\" (UID: \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:00.404866 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.404838    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:39:00.452092 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.452012    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:39:00.454843 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.454815    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w\" (UID: \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:00.454973 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.454920    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7vtw2\" (UniqueName: \"kubernetes.io/projected/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-kube-api-access-7vtw2\") pod \"payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w\" (UID: \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:00.455654 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.455629    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w\" (UID: \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:00.466718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.466667    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7vtw2\" (UniqueName: \"kubernetes.io/projected/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-kube-api-access-7vtw2\") pod \"payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w\" (UID: \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:00.539248 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.539122    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:00.616258 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.616222    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:00.730083 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.729999    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"]
Sep 10 14:39:00.734703 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.734680    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.736673 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.736650    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-e8a6c27c-serving-cert\""
Sep 10 14:39:00.741297 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.741277    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"]
Sep 10 14:39:00.747476 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.747450    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"]
Sep 10 14:39:00.749142 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:39:00.749113    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a334e0_45af_47e8_b6ca_33c1713d72e9.slice/crio-2daa72e0e43f16bb728def86bae3d0de7094a4a6c348df5ee9f0a42c92737446 WatchSource:0}: Error finding container 2daa72e0e43f16bb728def86bae3d0de7094a4a6c348df5ee9f0a42c92737446: Status 404 returned error can't find the container with id 2daa72e0e43f16bb728def86bae3d0de7094a4a6c348df5ee9f0a42c92737446
Sep 10 14:39:00.757681 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.757592    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.758175 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.758001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-metrics-certs\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.758175 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.758097    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86502cce-6ccc-455e-a4d4-ae4341b9ef47-openshift-service-ca\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.758450 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.758189    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jwnj\" (UniqueName: \"kubernetes.io/projected/86502cce-6ccc-455e-a4d4-ae4341b9ef47-kube-api-access-8jwnj\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.776213 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.776181    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:00.852455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.852415    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:00.859165 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.859143    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.859286 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.859172    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-metrics-certs\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.859286 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.859198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86502cce-6ccc-455e-a4d4-ae4341b9ef47-openshift-service-ca\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.859286 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.859232    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jwnj\" (UniqueName: \"kubernetes.io/projected/86502cce-6ccc-455e-a4d4-ae4341b9ef47-kube-api-access-8jwnj\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.859451 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:39:00.859316    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-serving-cert: secret "maas-api-e2e-ait-e8a6c27c-serving-cert" not found
Sep 10 14:39:00.859451 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:39:00.859385    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls podName:86502cce-6ccc-455e-a4d4-ae4341b9ef47 nodeName:}" failed. No retries permitted until 2026-09-10 14:39:01.359367937 +0000 UTC m=+1147.127090695 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls") pod "maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2" (UID: "86502cce-6ccc-455e-a4d4-ae4341b9ef47") : secret "maas-api-e2e-ait-e8a6c27c-serving-cert" not found
Sep 10 14:39:00.859922 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.859899    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86502cce-6ccc-455e-a4d4-ae4341b9ef47-openshift-service-ca\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.861715 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.861685    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-metrics-certs\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.867582 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.867563    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jwnj\" (UniqueName: \"kubernetes.io/projected/86502cce-6ccc-455e-a4d4-ae4341b9ef47-kube-api-access-8jwnj\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:00.890226 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.890201    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"]
Sep 10 14:39:00.893081 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:39:00.893048    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b8b50d6_1021_4d3a_baa7_01ab6afb1e10.slice/crio-6d395cf3901512b77514ecf3532115b5af87eabf72172d382f7b20ebd5b37a0f WatchSource:0}: Error finding container 6d395cf3901512b77514ecf3532115b5af87eabf72172d382f7b20ebd5b37a0f: Status 404 returned error can't find the container with id 6d395cf3901512b77514ecf3532115b5af87eabf72172d382f7b20ebd5b37a0f
Sep 10 14:39:00.964901 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:00.964860    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:01.364202 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.364126    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:01.366819 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.366791    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls\") pod \"maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:01.427625 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.427587    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:01.432609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.432585    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp" event={"ID":"64a334e0-45af-47e8-b6ca-33c1713d72e9","Type":"ContainerStarted","Data":"e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313"}
Sep 10 14:39:01.432735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.432663    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp" event={"ID":"64a334e0-45af-47e8-b6ca-33c1713d72e9","Type":"ContainerStarted","Data":"2daa72e0e43f16bb728def86bae3d0de7094a4a6c348df5ee9f0a42c92737446"}
Sep 10 14:39:01.432735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.432697    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:39:01.434542 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.434513    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w" event={"ID":"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10","Type":"ContainerStarted","Data":"bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84"}
Sep 10 14:39:01.434661 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.434573    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:01.434661 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.434595    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w" event={"ID":"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10","Type":"ContainerStarted","Data":"6d395cf3901512b77514ecf3532115b5af87eabf72172d382f7b20ebd5b37a0f"}
Sep 10 14:39:01.477330 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.477284    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp" podStartSLOduration=2.135641417 podStartE2EDuration="2.477269292s" totalImagesPullingTime="341.627875ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:38:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:01.474119974 +0000 UTC m=+1147.241842754" watchObservedRunningTime="2026-09-10 14:39:01.477269292 +0000 UTC m=+1147.244992071"
Sep 10 14:39:01.580863 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.580827    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:01.649943 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.649865    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:01.922254 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.922148    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w" podStartSLOduration=1.922122958 podStartE2EDuration="1.922122958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:01.506894445 +0000 UTC m=+1147.274617224" watchObservedRunningTime="2026-09-10 14:39:01.922122958 +0000 UTC m=+1147.689845738"
Sep 10 14:39:01.926413 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.924168    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"]
Sep 10 14:39:01.927902 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:39:01.927876    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86502cce_6ccc_455e_a4d4_ae4341b9ef47.slice/crio-d0dab3e187854815c6e7cc168f25ffd6697046596591dbc0bc6c95da668b70fb WatchSource:0}: Error finding container d0dab3e187854815c6e7cc168f25ffd6697046596591dbc0bc6c95da668b70fb: Status 404 returned error can't find the container with id d0dab3e187854815c6e7cc168f25ffd6697046596591dbc0bc6c95da668b70fb
Sep 10 14:39:01.979316 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:01.979283    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:02.441029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:02.440994    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2" event={"ID":"86502cce-6ccc-455e-a4d4-ae4341b9ef47","Type":"ContainerStarted","Data":"86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505"}
Sep 10 14:39:02.441205 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:02.441047    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2" event={"ID":"86502cce-6ccc-455e-a4d4-ae4341b9ef47","Type":"ContainerStarted","Data":"d0dab3e187854815c6e7cc168f25ffd6697046596591dbc0bc6c95da668b70fb"}
Sep 10 14:39:02.441205 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:02.441160    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:02.459639 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:02.459587    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2" podStartSLOduration=2.135422122 podStartE2EDuration="2.459570469s" totalImagesPullingTime="324.148347ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:02.458284083 +0000 UTC m=+1148.226006863" watchObservedRunningTime="2026-09-10 14:39:02.459570469 +0000 UTC m=+1148.227293249"
Sep 10 14:39:02.780309 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:02.780275    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:06.688060 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:06.688023    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:06.856471 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:06.856415    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:06.948325 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:06.948229    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:07.046564 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:07.046529    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:07.446550 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:07.446514    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:39:08.452208 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:08.452179    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:09.768332 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:09.768294    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:11.298353 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:11.298314    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:11.397394 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:11.397359    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:11.499189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:11.499158    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:12.443472 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:12.443440    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:39:13.689304 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:13.689268    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:15.087502 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:15.087465    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:16.224511 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:16.224475    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:16.605814 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:16.605779    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:16.869619 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:16.869523    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:16.946067 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:16.946027    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:17.951124 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:17.951087    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:18.051845 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:18.051806    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:18.151051 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:18.151011    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:18.285096 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:18.285007    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:20.405276 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:20.405244    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:20.528421 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:20.528391    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:20.731340 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:20.731263    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:22.122262 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.122224    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"]
Sep 10 14:39:22.125797 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.125775    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.127809 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.127787    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-2365fe-serving-cert\""
Sep 10 14:39:22.135926 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.135906    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"]
Sep 10 14:39:22.162186 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.162154    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:22.261762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.261714    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.261954 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.261837    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.262030 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.261977    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-metrics-certs\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.262030 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.262007    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whjrf\" (UniqueName: \"kubernetes.io/projected/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-kube-api-access-whjrf\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.262798 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.262768    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:22.362625 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.362592    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.362810 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.362655    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.362810 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.362728    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-metrics-certs\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.362810 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.362770    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whjrf\" (UniqueName: \"kubernetes.io/projected/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-kube-api-access-whjrf\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.362937 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:39:22.362852    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-serving-cert: secret "maas-api-e2e-inf-b-2365fe-serving-cert" not found
Sep 10 14:39:22.362968 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:39:22.362937    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls podName:04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed nodeName:}" failed. No retries permitted until 2026-09-10 14:39:22.862917034 +0000 UTC m=+1168.630639809 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls") pod "maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd" (UID: "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed") : secret "maas-api-e2e-inf-b-2365fe-serving-cert" not found
Sep 10 14:39:22.363377 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.363353    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-openshift-service-ca\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.365526 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.365459    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-metrics-certs\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.371404 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.371377    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whjrf\" (UniqueName: \"kubernetes.io/projected/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-kube-api-access-whjrf\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.682077 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.682023    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:22.779830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.779790    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:22.854166 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.854132    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:22.867545 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.867517    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.868603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.868581    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"]
Sep 10 14:39:22.870043 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.870021    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls\") pod \"maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:22.872264 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.872245    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:22.874045 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.874025    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-2365fe-dockercfg-brl6s\""
Sep 10 14:39:22.874498 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.874085    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-2365fe\""
Sep 10 14:39:22.880815 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.880793    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"]
Sep 10 14:39:22.968230 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.968150    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80010163-72d3-4558-9a49-06822081e1b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7\" (UID: \"80010163-72d3-4558-9a49-06822081e1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:22.968230 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:22.968225    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klvsh\" (UniqueName: \"kubernetes.io/projected/80010163-72d3-4558-9a49-06822081e1b9-kube-api-access-klvsh\") pod \"payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7\" (UID: \"80010163-72d3-4558-9a49-06822081e1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:23.038932 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.038897    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:23.069396 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.069358    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80010163-72d3-4558-9a49-06822081e1b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7\" (UID: \"80010163-72d3-4558-9a49-06822081e1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:23.069552 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.069445    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klvsh\" (UniqueName: \"kubernetes.io/projected/80010163-72d3-4558-9a49-06822081e1b9-kube-api-access-klvsh\") pod \"payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7\" (UID: \"80010163-72d3-4558-9a49-06822081e1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:23.070294 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.070266    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80010163-72d3-4558-9a49-06822081e1b9-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7\" (UID: \"80010163-72d3-4558-9a49-06822081e1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:23.077174 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.077150    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klvsh\" (UniqueName: \"kubernetes.io/projected/80010163-72d3-4558-9a49-06822081e1b9-kube-api-access-klvsh\") pod \"payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7\" (UID: \"80010163-72d3-4558-9a49-06822081e1b9\") " pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:23.141640 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.141606    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:23.191002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.190973    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:23.321384 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.321359    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"]
Sep 10 14:39:23.322830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.322798    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:23.323548 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:39:23.323519    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04bc9d14_100f_42e2_a0fc_d0b6fdfd0eed.slice/crio-1bfd02c89c235393ae9815a40ab7f1b1af2825b4cf2dd970b2d69099ab8fc872 WatchSource:0}: Error finding container 1bfd02c89c235393ae9815a40ab7f1b1af2825b4cf2dd970b2d69099ab8fc872: Status 404 returned error can't find the container with id 1bfd02c89c235393ae9815a40ab7f1b1af2825b4cf2dd970b2d69099ab8fc872
Sep 10 14:39:23.461680 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.461645    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"]
Sep 10 14:39:23.465007 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:39:23.464974    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80010163_72d3_4558_9a49_06822081e1b9.slice/crio-f9190425b3d66cb8733c627bb7f62bc7913f26957b6533b0e8f289979d64dc71 WatchSource:0}: Error finding container f9190425b3d66cb8733c627bb7f62bc7913f26957b6533b0e8f289979d64dc71: Status 404 returned error can't find the container with id f9190425b3d66cb8733c627bb7f62bc7913f26957b6533b0e8f289979d64dc71
Sep 10 14:39:23.553773 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.553716    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7" event={"ID":"80010163-72d3-4558-9a49-06822081e1b9","Type":"ContainerStarted","Data":"7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79"}
Sep 10 14:39:23.554292 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.553915    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:23.554292 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.553955    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7" event={"ID":"80010163-72d3-4558-9a49-06822081e1b9","Type":"ContainerStarted","Data":"f9190425b3d66cb8733c627bb7f62bc7913f26957b6533b0e8f289979d64dc71"}
Sep 10 14:39:23.555007 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.554986    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd" event={"ID":"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed","Type":"ContainerStarted","Data":"1bfd02c89c235393ae9815a40ab7f1b1af2825b4cf2dd970b2d69099ab8fc872"}
Sep 10 14:39:23.569265 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:23.569207    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7" podStartSLOduration=1.569188445 podStartE2EDuration="1.569188445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:23.568874974 +0000 UTC m=+1169.336597754" watchObservedRunningTime="2026-09-10 14:39:23.569188445 +0000 UTC m=+1169.336911225"
Sep 10 14:39:24.076846 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:24.076730    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:24.562741 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:24.562639    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd" event={"ID":"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed","Type":"ContainerStarted","Data":"919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14"}
Sep 10 14:39:24.562741 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:24.562717    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:24.579262 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:24.579203    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd" podStartSLOduration=2.105344101 podStartE2EDuration="2.579186313s" totalImagesPullingTime="473.842212ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:24.577180668 +0000 UTC m=+1170.344903447" watchObservedRunningTime="2026-09-10 14:39:24.579186313 +0000 UTC m=+1170.346909084"
Sep 10 14:39:24.782173 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:24.782131    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:24.883485 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:24.883451    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:25.533490 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:25.533455    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:30.576493 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:30.576461    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:39:34.565275 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:34.565238    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:39:40.390616 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.390582    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"]
Sep 10 14:39:40.391123 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.390829    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2" podUID="86502cce-6ccc-455e-a4d4-ae4341b9ef47" containerName="maas-api" containerID="cri-o://86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505" gracePeriod=30
Sep 10 14:39:40.419982 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.419959    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:40.493105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.493066    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:40.570900 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.570863    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:40.636246 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.636218    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:40.667322 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.667213    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:40.667861 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.667834    2578 generic.go:356] "Generic (PLEG): container finished" podID="86502cce-6ccc-455e-a4d4-ae4341b9ef47" containerID="86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505" exitCode=0
Sep 10 14:39:40.668023 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.667877    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2" event={"ID":"86502cce-6ccc-455e-a4d4-ae4341b9ef47","Type":"ContainerDied","Data":"86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505"}
Sep 10 14:39:40.668023 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.667914    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"
Sep 10 14:39:40.668023 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.667934    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2" event={"ID":"86502cce-6ccc-455e-a4d4-ae4341b9ef47","Type":"ContainerDied","Data":"d0dab3e187854815c6e7cc168f25ffd6697046596591dbc0bc6c95da668b70fb"}
Sep 10 14:39:40.668023 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.667967    2578 scope.go:117] "RemoveContainer" containerID="86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505"
Sep 10 14:39:40.678427 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.678407    2578 scope.go:117] "RemoveContainer" containerID="86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505"
Sep 10 14:39:40.678807 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:39:40.678784    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505\": container with ID starting with 86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505 not found: ID does not exist" containerID="86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505"
Sep 10 14:39:40.678869 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.678818    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505"} err="failed to get container status \"86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505\": rpc error: code = NotFound desc = could not find container \"86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505\": container with ID starting with 86bfda0fa7c09dde76fb914e4c627b0f48c57984072aa1cc8259346840caa505 not found: ID does not exist"
Sep 10 14:39:40.723022 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.723001    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8jwnj\" (UniqueName: \"kubernetes.io/projected/86502cce-6ccc-455e-a4d4-ae4341b9ef47-kube-api-access-8jwnj\") pod \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") "
Sep 10 14:39:40.723178 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.723048    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls\") pod \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") "
Sep 10 14:39:40.723178 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.723070    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86502cce-6ccc-455e-a4d4-ae4341b9ef47-openshift-service-ca\") pod \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") "
Sep 10 14:39:40.723178 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.723102    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-metrics-certs\") pod \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\" (UID: \"86502cce-6ccc-455e-a4d4-ae4341b9ef47\") "
Sep 10 14:39:40.723517 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.723490    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86502cce-6ccc-455e-a4d4-ae4341b9ef47-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "86502cce-6ccc-455e-a4d4-ae4341b9ef47" (UID: "86502cce-6ccc-455e-a4d4-ae4341b9ef47"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:39:40.725597 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.725571    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "86502cce-6ccc-455e-a4d4-ae4341b9ef47" (UID: "86502cce-6ccc-455e-a4d4-ae4341b9ef47"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:39:40.725855 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.725833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "86502cce-6ccc-455e-a4d4-ae4341b9ef47" (UID: "86502cce-6ccc-455e-a4d4-ae4341b9ef47"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:39:40.725929 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.725865    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86502cce-6ccc-455e-a4d4-ae4341b9ef47-kube-api-access-8jwnj" (OuterVolumeSpecName: "kube-api-access-8jwnj") pod "86502cce-6ccc-455e-a4d4-ae4341b9ef47" (UID: "86502cce-6ccc-455e-a4d4-ae4341b9ef47"). InnerVolumeSpecName "kube-api-access-8jwnj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:39:40.731350 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.731311    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:40.808517 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.808471    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:40.824667 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.824642    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:39:40.824667 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.824667    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jwnj\" (UniqueName: \"kubernetes.io/projected/86502cce-6ccc-455e-a4d4-ae4341b9ef47-kube-api-access-8jwnj\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:39:40.824874 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.824676    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/86502cce-6ccc-455e-a4d4-ae4341b9ef47-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:39:40.824874 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.824685    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/86502cce-6ccc-455e-a4d4-ae4341b9ef47-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:39:40.987607 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.987579    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"]
Sep 10 14:39:40.992403 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:40.992376    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-e8a6c27c-786d497d99-bsqk2"]
Sep 10 14:39:41.249733 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:41.249653    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:41.353903 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:41.353870    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:41.473037 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:41.472982    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:41.789040 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:41.788993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:41.908603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:41.908571    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:42.732225 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:42.732188    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86502cce-6ccc-455e-a4d4-ae4341b9ef47" path="/var/lib/kubelet/pods/86502cce-6ccc-455e-a4d4-ae4341b9ef47/volumes"
Sep 10 14:39:43.262690 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:43.262654    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:44.250161 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.250118    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:44.348631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.348597    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:44.540100 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.539975    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"]
Sep 10 14:39:44.540803 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.540614    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86502cce-6ccc-455e-a4d4-ae4341b9ef47" containerName="maas-api"
Sep 10 14:39:44.540803 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.540645    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="86502cce-6ccc-455e-a4d4-ae4341b9ef47" containerName="maas-api"
Sep 10 14:39:44.540803 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.540738    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="86502cce-6ccc-455e-a4d4-ae4341b9ef47" containerName="maas-api"
Sep 10 14:39:44.545521 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.545499    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.547929 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.547906    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-7e4309cc-serving-cert\""
Sep 10 14:39:44.557199 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.557176    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"]
Sep 10 14:39:44.571105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.571049    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:44.659891 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.659858    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-maas-api-tls\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.659891 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.659895    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-openshift-service-ca\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.660165 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.659924    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw89m\" (UniqueName: \"kubernetes.io/projected/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-kube-api-access-qw89m\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.660165 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.660001    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-metrics-certs\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.666738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.666706    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:44.760850 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.760807    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw89m\" (UniqueName: \"kubernetes.io/projected/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-kube-api-access-qw89m\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.760850 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.760861    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-metrics-certs\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.761105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.761089    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-maas-api-tls\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.761183 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.761126    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-openshift-service-ca\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.761670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.761649    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-openshift-service-ca\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.763355 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.763330    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-metrics-certs\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.763444 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.763336    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-maas-api-tls\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.768365 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.768344    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw89m\" (UniqueName: \"kubernetes.io/projected/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-kube-api-access-qw89m\") pod \"maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:44.860481 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:44.860451    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:45.142632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.142601    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"]
Sep 10 14:39:45.145294 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:39:45.145262    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5188ec07_5688_49ab_9a3e_58e5b28f5f9d.slice/crio-77b44498c30675af3f41b14e6667ef3198b68ef03f55691ed656fb4350fb08a4 WatchSource:0}: Error finding container 77b44498c30675af3f41b14e6667ef3198b68ef03f55691ed656fb4350fb08a4: Status 404 returned error can't find the container with id 77b44498c30675af3f41b14e6667ef3198b68ef03f55691ed656fb4350fb08a4
Sep 10 14:39:45.186292 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.186264    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:45.290838 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.290803    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:45.381843 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.381809    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:45.593243 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.593212    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:45.696805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.696763    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l" event={"ID":"5188ec07-5688-49ab-9a3e-58e5b28f5f9d","Type":"ContainerStarted","Data":"0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00"}
Sep 10 14:39:45.697002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.696824    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l" event={"ID":"5188ec07-5688-49ab-9a3e-58e5b28f5f9d","Type":"ContainerStarted","Data":"77b44498c30675af3f41b14e6667ef3198b68ef03f55691ed656fb4350fb08a4"}
Sep 10 14:39:45.697002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.696890    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:39:45.714556 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:45.714503    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l" podStartSLOduration=1.373097083 podStartE2EDuration="1.714486923s" totalImagesPullingTime="341.38984ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:39:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:39:45.712866496 +0000 UTC m=+1191.480589276" watchObservedRunningTime="2026-09-10 14:39:45.714486923 +0000 UTC m=+1191.482209704"
Sep 10 14:39:46.531022 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:46.530982    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:50.055613 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:50.055585    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:39:51.707651 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:39:51.707620    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:40:07.747952 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:07.747910    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:15.745492 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:15.745458    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"]
Sep 10 14:40:15.748245 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:15.745920    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l" podUID="5188ec07-5688-49ab-9a3e-58e5b28f5f9d" containerName="maas-api" containerID="cri-o://0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00" gracePeriod=30
Sep 10 14:40:15.773188 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:15.773153    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:15.839120 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:15.839076    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:15.903897 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:15.903863    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:15.977941 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:15.977892    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:15.999158 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:15.999096    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:40:16.043550 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.043513    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:16.135930 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.135894    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:16.148026 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.147998    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qw89m\" (UniqueName: \"kubernetes.io/projected/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-kube-api-access-qw89m\") pod \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") "
Sep 10 14:40:16.148195 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.148061    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-openshift-service-ca\") pod \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") "
Sep 10 14:40:16.148195 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.148084    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-metrics-certs\") pod \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") "
Sep 10 14:40:16.148195 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.148186    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-maas-api-tls\") pod \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\" (UID: \"5188ec07-5688-49ab-9a3e-58e5b28f5f9d\") "
Sep 10 14:40:16.148417 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.148384    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5188ec07-5688-49ab-9a3e-58e5b28f5f9d" (UID: "5188ec07-5688-49ab-9a3e-58e5b28f5f9d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:40:16.148580 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.148561    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:16.150265 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.150245    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "5188ec07-5688-49ab-9a3e-58e5b28f5f9d" (UID: "5188ec07-5688-49ab-9a3e-58e5b28f5f9d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:40:16.150670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.150650    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5188ec07-5688-49ab-9a3e-58e5b28f5f9d" (UID: "5188ec07-5688-49ab-9a3e-58e5b28f5f9d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:40:16.150794 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.150686    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-kube-api-access-qw89m" (OuterVolumeSpecName: "kube-api-access-qw89m") pod "5188ec07-5688-49ab-9a3e-58e5b28f5f9d" (UID: "5188ec07-5688-49ab-9a3e-58e5b28f5f9d"). InnerVolumeSpecName "kube-api-access-qw89m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:40:16.250007 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.249936    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:16.250007 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.249960    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qw89m\" (UniqueName: \"kubernetes.io/projected/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-kube-api-access-qw89m\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:16.250007 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.249970    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5188ec07-5688-49ab-9a3e-58e5b28f5f9d-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:16.557258 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.557173    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:16.862861 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.862823    2578 generic.go:356] "Generic (PLEG): container finished" podID="5188ec07-5688-49ab-9a3e-58e5b28f5f9d" containerID="0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00" exitCode=0
Sep 10 14:40:16.863347 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.862884    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"
Sep 10 14:40:16.863347 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.862889    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l" event={"ID":"5188ec07-5688-49ab-9a3e-58e5b28f5f9d","Type":"ContainerDied","Data":"0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00"}
Sep 10 14:40:16.863347 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.863054    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:16.863347 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.863141    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l" event={"ID":"5188ec07-5688-49ab-9a3e-58e5b28f5f9d","Type":"ContainerDied","Data":"77b44498c30675af3f41b14e6667ef3198b68ef03f55691ed656fb4350fb08a4"}
Sep 10 14:40:16.863347 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.863192    2578 scope.go:117] "RemoveContainer" containerID="0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00"
Sep 10 14:40:16.876075 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.876056    2578 scope.go:117] "RemoveContainer" containerID="0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00"
Sep 10 14:40:16.876409 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:40:16.876387    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00\": container with ID starting with 0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00 not found: ID does not exist" containerID="0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00"
Sep 10 14:40:16.876465 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.876418    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00"} err="failed to get container status \"0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00\": rpc error: code = NotFound desc = could not find container \"0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00\": container with ID starting with 0a1f79cd8d043b6ce4f52b8bd4a4df48438f68ae815c3f8fdd1cfecc65cd4e00 not found: ID does not exist"
Sep 10 14:40:16.880934 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.880913    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"]
Sep 10 14:40:16.884007 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:16.883985    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-7e4309cc-7f4688bd8f-7jw2l"]
Sep 10 14:40:17.075667 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:17.075624    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:18.724985 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:18.724953    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5188ec07-5688-49ab-9a3e-58e5b28f5f9d" path="/var/lib/kubelet/pods/5188ec07-5688-49ab-9a3e-58e5b28f5f9d/volumes"
Sep 10 14:40:19.404535 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:19.404495    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:19.501474 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:19.501443    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:20.534974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.534944    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"]
Sep 10 14:40:20.535572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.535556    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5188ec07-5688-49ab-9a3e-58e5b28f5f9d" containerName="maas-api"
Sep 10 14:40:20.535615 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.535578    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5188ec07-5688-49ab-9a3e-58e5b28f5f9d" containerName="maas-api"
Sep 10 14:40:20.535707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.535695    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5188ec07-5688-49ab-9a3e-58e5b28f5f9d" containerName="maas-api"
Sep 10 14:40:20.540297 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.540278    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.542364 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.542347    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-04dfc68a-serving-cert\""
Sep 10 14:40:20.549223 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.549204    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"]
Sep 10 14:40:20.587031 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.586984    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.587214 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.587129    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-metrics-certs\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.587214 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.587163    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5cjb\" (UniqueName: \"kubernetes.io/projected/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-kube-api-access-z5cjb\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.587317 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.587278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-openshift-service-ca\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.588019 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.587988    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:20.688463 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.688427    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-metrics-certs\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.688463 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.688464    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5cjb\" (UniqueName: \"kubernetes.io/projected/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-kube-api-access-z5cjb\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.688707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.688527    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-openshift-service-ca\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.688707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.688544    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.688707 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:40:20.688645    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-serving-cert: secret "maas-api-e2e-ait-04dfc68a-serving-cert" not found
Sep 10 14:40:20.688707 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:40:20.688703    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls podName:cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d nodeName:}" failed. No retries permitted until 2026-09-10 14:40:21.188688109 +0000 UTC m=+1226.956410886 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls") pod "maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9" (UID: "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d") : secret "maas-api-e2e-ait-04dfc68a-serving-cert" not found
Sep 10 14:40:20.689389 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.689362    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:20.689494 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.689467    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-openshift-service-ca\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.691080 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.691062    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-metrics-certs\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.699174 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.699151    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5cjb\" (UniqueName: \"kubernetes.io/projected/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-kube-api-access-z5cjb\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:20.844961 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:20.844921    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:21.172429 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:21.172340    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:21.192640 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:21.192601    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:21.195077 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:21.195054    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls\") pod \"maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:21.278926 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:21.278876    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:21.452791 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:21.452685    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:21.728103 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:21.728081    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"]
Sep 10 14:40:21.729846 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:40:21.729809    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd7ed3a2_8f57_4910_ad3a_f418b8c69b8d.slice/crio-5190653ad459b21deb89d7154da53da7771ba88f856c122c73d6efce50bad0ec WatchSource:0}: Error finding container 5190653ad459b21deb89d7154da53da7771ba88f856c122c73d6efce50bad0ec: Status 404 returned error can't find the container with id 5190653ad459b21deb89d7154da53da7771ba88f856c122c73d6efce50bad0ec
Sep 10 14:40:21.892656 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:21.892625    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9" event={"ID":"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d","Type":"ContainerStarted","Data":"5190653ad459b21deb89d7154da53da7771ba88f856c122c73d6efce50bad0ec"}
Sep 10 14:40:22.898754 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:22.898715    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9" event={"ID":"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d","Type":"ContainerStarted","Data":"2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af"}
Sep 10 14:40:22.899179 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:22.898799    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:22.922772 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:22.922712    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9" podStartSLOduration=2.570627248 podStartE2EDuration="2.922696201s" totalImagesPullingTime="352.068953ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:22.919930535 +0000 UTC m=+1228.687653314" watchObservedRunningTime="2026-09-10 14:40:22.922696201 +0000 UTC m=+1228.690418980"
Sep 10 14:40:25.013624 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:25.013589    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:27.265998 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:27.265955    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:28.910405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:28.910372    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:40.514257 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:40.514205    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:41.207688 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:41.207653    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:41.954957 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:41.954920    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:42.676980 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:42.676930    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:43.379159 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:43.379129    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:43.494040 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:43.494002    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:47.962542 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:47.962455    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:49.701575 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:49.701543    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.146874 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.146832    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"]
Sep 10 14:40:51.147350 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.147218    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp" podUID="64a334e0-45af-47e8-b6ca-33c1713d72e9" containerName="maas-api" containerID="cri-o://e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313" gracePeriod=30
Sep 10 14:40:51.181527 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.181501    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.225351 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.225317    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"]
Sep 10 14:40:51.225575 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.225548    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w" podUID="9b8b50d6-1021-4d3a-baa7-01ab6afb1e10" containerName="payload-pre-processing" containerID="cri-o://bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84" gracePeriod=30
Sep 10 14:40:51.295728 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.295689    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.340490 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.340465    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"]
Sep 10 14:40:51.340724 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.340692    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9" podUID="cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" containerName="maas-api" containerID="cri-o://2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af" gracePeriod=30
Sep 10 14:40:51.398835 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.398720    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.502144 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.502055    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.528429 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.528403    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:40:51.542681 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.542657    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:40:51.624998 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.624974    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:51.626976 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.626948    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.672285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672206    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-plugins-config-volume\") pod \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\" (UID: \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\") "
Sep 10 14:40:51.672285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672274    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls\") pod \"64a334e0-45af-47e8-b6ca-33c1713d72e9\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") "
Sep 10 14:40:51.672501 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672290    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7vtw2\" (UniqueName: \"kubernetes.io/projected/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-kube-api-access-7vtw2\") pod \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\" (UID: \"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10\") "
Sep 10 14:40:51.672501 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672314    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntq5f\" (UniqueName: \"kubernetes.io/projected/64a334e0-45af-47e8-b6ca-33c1713d72e9-kube-api-access-ntq5f\") pod \"64a334e0-45af-47e8-b6ca-33c1713d72e9\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") "
Sep 10 14:40:51.672501 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672433    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-metrics-certs\") pod \"64a334e0-45af-47e8-b6ca-33c1713d72e9\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") "
Sep 10 14:40:51.672663 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672575    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64a334e0-45af-47e8-b6ca-33c1713d72e9-openshift-service-ca\") pod \"64a334e0-45af-47e8-b6ca-33c1713d72e9\" (UID: \"64a334e0-45af-47e8-b6ca-33c1713d72e9\") "
Sep 10 14:40:51.672715 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672654    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9b8b50d6-1021-4d3a-baa7-01ab6afb1e10" (UID: "9b8b50d6-1021-4d3a-baa7-01ab6afb1e10"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:40:51.672989 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672964    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.672989 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.672967    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64a334e0-45af-47e8-b6ca-33c1713d72e9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "64a334e0-45af-47e8-b6ca-33c1713d72e9" (UID: "64a334e0-45af-47e8-b6ca-33c1713d72e9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:40:51.674769 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.674717    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "64a334e0-45af-47e8-b6ca-33c1713d72e9" (UID: "64a334e0-45af-47e8-b6ca-33c1713d72e9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:40:51.675059 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.675032    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64a334e0-45af-47e8-b6ca-33c1713d72e9-kube-api-access-ntq5f" (OuterVolumeSpecName: "kube-api-access-ntq5f") pod "64a334e0-45af-47e8-b6ca-33c1713d72e9" (UID: "64a334e0-45af-47e8-b6ca-33c1713d72e9"). InnerVolumeSpecName "kube-api-access-ntq5f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:40:51.675059 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.675046    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "64a334e0-45af-47e8-b6ca-33c1713d72e9" (UID: "64a334e0-45af-47e8-b6ca-33c1713d72e9"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:40:51.675187 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.675061    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-kube-api-access-7vtw2" (OuterVolumeSpecName: "kube-api-access-7vtw2") pod "9b8b50d6-1021-4d3a-baa7-01ab6afb1e10" (UID: "9b8b50d6-1021-4d3a-baa7-01ab6afb1e10"). InnerVolumeSpecName "kube-api-access-7vtw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:40:51.727992 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.727964    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.774262 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774234    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-metrics-certs\") pod \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") "
Sep 10 14:40:51.774387 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774276    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5cjb\" (UniqueName: \"kubernetes.io/projected/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-kube-api-access-z5cjb\") pod \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") "
Sep 10 14:40:51.774444 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774399    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls\") pod \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") "
Sep 10 14:40:51.774496 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774452    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-openshift-service-ca\") pod \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\" (UID: \"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d\") "
Sep 10 14:40:51.774852 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774805    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.774852 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774829    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7vtw2\" (UniqueName: \"kubernetes.io/projected/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10-kube-api-access-7vtw2\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.774852 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774843    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntq5f\" (UniqueName: \"kubernetes.io/projected/64a334e0-45af-47e8-b6ca-33c1713d72e9-kube-api-access-ntq5f\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.774852 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774856    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/64a334e0-45af-47e8-b6ca-33c1713d72e9-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.774852 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774869    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/64a334e0-45af-47e8-b6ca-33c1713d72e9-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.775355 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.774887    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" (UID: "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:40:51.776505 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.776477    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-kube-api-access-z5cjb" (OuterVolumeSpecName: "kube-api-access-z5cjb") pod "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" (UID: "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d"). InnerVolumeSpecName "kube-api-access-z5cjb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:40:51.776595 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.776543    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" (UID: "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:40:51.776786 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.776768    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" (UID: "cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:40:51.828728 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.828690    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:51.876031 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.875995    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.876031 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.876031    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5cjb\" (UniqueName: \"kubernetes.io/projected/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-kube-api-access-z5cjb\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.876216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.876045    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:51.876216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:51.876058    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:40:52.048904 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.048819    2578 generic.go:356] "Generic (PLEG): container finished" podID="64a334e0-45af-47e8-b6ca-33c1713d72e9" containerID="e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313" exitCode=0
Sep 10 14:40:52.048904 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.048882    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"
Sep 10 14:40:52.049114 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.048907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp" event={"ID":"64a334e0-45af-47e8-b6ca-33c1713d72e9","Type":"ContainerDied","Data":"e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313"}
Sep 10 14:40:52.049114 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.048962    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp" event={"ID":"64a334e0-45af-47e8-b6ca-33c1713d72e9","Type":"ContainerDied","Data":"2daa72e0e43f16bb728def86bae3d0de7094a4a6c348df5ee9f0a42c92737446"}
Sep 10 14:40:52.049114 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.048990    2578 scope.go:117] "RemoveContainer" containerID="e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313"
Sep 10 14:40:52.050295 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.050274    2578 generic.go:356] "Generic (PLEG): container finished" podID="9b8b50d6-1021-4d3a-baa7-01ab6afb1e10" containerID="bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84" exitCode=0
Sep 10 14:40:52.050393 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.050320    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"
Sep 10 14:40:52.050393 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.050352    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w" event={"ID":"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10","Type":"ContainerDied","Data":"bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84"}
Sep 10 14:40:52.050498 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.050400    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w" event={"ID":"9b8b50d6-1021-4d3a-baa7-01ab6afb1e10","Type":"ContainerDied","Data":"6d395cf3901512b77514ecf3532115b5af87eabf72172d382f7b20ebd5b37a0f"}
Sep 10 14:40:52.051918 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.051879    2578 generic.go:356] "Generic (PLEG): container finished" podID="cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" containerID="2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af" exitCode=0
Sep 10 14:40:52.052026 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.051923    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"
Sep 10 14:40:52.052026 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.051969    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9" event={"ID":"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d","Type":"ContainerDied","Data":"2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af"}
Sep 10 14:40:52.052147 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.052017    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9" event={"ID":"cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d","Type":"ContainerDied","Data":"5190653ad459b21deb89d7154da53da7771ba88f856c122c73d6efce50bad0ec"}
Sep 10 14:40:52.059344 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.059328    2578 scope.go:117] "RemoveContainer" containerID="e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313"
Sep 10 14:40:52.059635 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:40:52.059618    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313\": container with ID starting with e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313 not found: ID does not exist" containerID="e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313"
Sep 10 14:40:52.059712 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.059641    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313"} err="failed to get container status \"e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313\": rpc error: code = NotFound desc = could not find container \"e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313\": container with ID starting with e3095f0f1c367bb398d8c8b776a367c7118b561ff3dcad8626d9f9b6da8dd313 not found: ID does not exist"
Sep 10 14:40:52.059712 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.059657    2578 scope.go:117] "RemoveContainer" containerID="bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84"
Sep 10 14:40:52.069679 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.069655    2578 scope.go:117] "RemoveContainer" containerID="bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84"
Sep 10 14:40:52.070005 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:40:52.069985    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84\": container with ID starting with bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84 not found: ID does not exist" containerID="bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84"
Sep 10 14:40:52.070081 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.070011    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84"} err="failed to get container status \"bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84\": rpc error: code = NotFound desc = could not find container \"bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84\": container with ID starting with bd56c0d1758889aaad2265f82e807726c64b05affce62750311a54ee31253a84 not found: ID does not exist"
Sep 10 14:40:52.070081 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.070027    2578 scope.go:117] "RemoveContainer" containerID="2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af"
Sep 10 14:40:52.077830 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.077815    2578 scope.go:117] "RemoveContainer" containerID="2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af"
Sep 10 14:40:52.078144 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:40:52.078125    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af\": container with ID starting with 2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af not found: ID does not exist" containerID="2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af"
Sep 10 14:40:52.078225 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.078150    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af"} err="failed to get container status \"2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af\": rpc error: code = NotFound desc = could not find container \"2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af\": container with ID starting with 2b6cbdd7531c9b3afdc0dc5d11e0ea33b915b604ce90f6dc95f129aa17b290af not found: ID does not exist"
Sep 10 14:40:52.079538 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.079512    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"]
Sep 10 14:40:52.085571 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.085545    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-e7415a-557d475db6-7htsp"]
Sep 10 14:40:52.095726 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.095697    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"]
Sep 10 14:40:52.097966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.097942    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-04dfc68a-75776d5bf6-hnzs9"]
Sep 10 14:40:52.107137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.107114    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"]
Sep 10 14:40:52.115941 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.115919    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-e7415a-565ff8fd74-w2k5w"]
Sep 10 14:40:52.724190 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.724157    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64a334e0-45af-47e8-b6ca-33c1713d72e9" path="/var/lib/kubelet/pods/64a334e0-45af-47e8-b6ca-33c1713d72e9/volumes"
Sep 10 14:40:52.724560 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.724546    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9b8b50d6-1021-4d3a-baa7-01ab6afb1e10" path="/var/lib/kubelet/pods/9b8b50d6-1021-4d3a-baa7-01ab6afb1e10/volumes"
Sep 10 14:40:52.724946 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.724932    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" path="/var/lib/kubelet/pods/cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d/volumes"
Sep 10 14:40:52.826555 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:52.826521    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:53.193061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:53.193028    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:53.479689 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:53.479612    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:53.805499 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:53.805419    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:55.601169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:55.601126    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:55.721381 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:55.721333    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:56.683266 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683236    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"]
Sep 10 14:40:56.683636 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683624    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9b8b50d6-1021-4d3a-baa7-01ab6afb1e10" containerName="payload-pre-processing"
Sep 10 14:40:56.683683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683637    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9b8b50d6-1021-4d3a-baa7-01ab6afb1e10" containerName="payload-pre-processing"
Sep 10 14:40:56.683683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683648    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64a334e0-45af-47e8-b6ca-33c1713d72e9" containerName="maas-api"
Sep 10 14:40:56.683683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683653    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="64a334e0-45af-47e8-b6ca-33c1713d72e9" containerName="maas-api"
Sep 10 14:40:56.683683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683673    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" containerName="maas-api"
Sep 10 14:40:56.683683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683678    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" containerName="maas-api"
Sep 10 14:40:56.683981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683793    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd7ed3a2-8f57-4910-ad3a-f418b8c69b8d" containerName="maas-api"
Sep 10 14:40:56.683981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683809    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9b8b50d6-1021-4d3a-baa7-01ab6afb1e10" containerName="payload-pre-processing"
Sep 10 14:40:56.683981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.683822    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="64a334e0-45af-47e8-b6ca-33c1713d72e9" containerName="maas-api"
Sep 10 14:40:56.688904 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.688883    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.690941 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.690918    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-b46e5fd4-serving-cert\""
Sep 10 14:40:56.696624 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.696600    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"]
Sep 10 14:40:56.724342 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.724305    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:56.823543 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.823507    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlzr9\" (UniqueName: \"kubernetes.io/projected/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-kube-api-access-rlzr9\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.823681 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.823573    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-maas-api-tls\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.823681 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.823627    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-openshift-service-ca\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.823681 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.823654    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-metrics-certs\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.924227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.924193    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlzr9\" (UniqueName: \"kubernetes.io/projected/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-kube-api-access-rlzr9\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.924389 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.924258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-maas-api-tls\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.924471 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.924443    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-openshift-service-ca\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.924535 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.924512    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-metrics-certs\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.924852 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.924822    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:56.925171 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.925149    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-openshift-service-ca\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.926898 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.926879    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-maas-api-tls\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.927164 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.927144    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-metrics-certs\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:56.932042 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:56.932020    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlzr9\" (UniqueName: \"kubernetes.io/projected/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-kube-api-access-rlzr9\") pod \"maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:57.002637 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:57.002571    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:57.027505 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:57.027475    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:57.224974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:57.224942    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:57.278189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:57.278165    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"]
Sep 10 14:40:57.280953 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:40:57.280928    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7afb576_cc8a_4bed_a2a4_7936bb92d23b.slice/crio-5d51316e51283c0a95d54a6b076ae7a73cbd7fcc3e836b1916b4af23e28fe0bb WatchSource:0}: Error finding container 5d51316e51283c0a95d54a6b076ae7a73cbd7fcc3e836b1916b4af23e28fe0bb: Status 404 returned error can't find the container with id 5d51316e51283c0a95d54a6b076ae7a73cbd7fcc3e836b1916b4af23e28fe0bb
Sep 10 14:40:57.324080 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:57.324046    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:40:58.091132 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:58.091095    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4" event={"ID":"f7afb576-cc8a-4bed-a2a4-7936bb92d23b","Type":"ContainerStarted","Data":"97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b"}
Sep 10 14:40:58.091573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:58.091152    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4" event={"ID":"f7afb576-cc8a-4bed-a2a4-7936bb92d23b","Type":"ContainerStarted","Data":"5d51316e51283c0a95d54a6b076ae7a73cbd7fcc3e836b1916b4af23e28fe0bb"}
Sep 10 14:40:58.091573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:58.091193    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:40:58.106971 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:40:58.106926    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4" podStartSLOduration=1.405348847 podStartE2EDuration="2.106910583s" totalImagesPullingTime="701.561736ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:40:58.104560114 +0000 UTC m=+1263.872282893" watchObservedRunningTime="2026-09-10 14:40:58.106910583 +0000 UTC m=+1263.874633362"
Sep 10 14:41:02.680977 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:02.680937    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:02.934366 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:02.934274    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:03.357092 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:03.357056    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:03.658152 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:03.658062    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:04.059428 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:04.059346    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:04.101944 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:04.101918    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:41:07.093889 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:07.093854    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:07.193434 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:07.193399    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:08.320763 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.320708    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"]
Sep 10 14:41:08.325575 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.325550    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.327351 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.327330    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-explicit-b2dfe7-serving-cert\""
Sep 10 14:41:08.335974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.335954    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"]
Sep 10 14:41:08.357624 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.357590    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:08.420519 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.420489    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:08.432139 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.432107    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f24bcba-cf62-4980-9875-af536f59a8d3-openshift-service-ca\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.432251 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.432171    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8mzr\" (UniqueName: \"kubernetes.io/projected/7f24bcba-cf62-4980-9875-af536f59a8d3-kube-api-access-h8mzr\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.432251 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.432217    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-metrics-certs\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.432339 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.432309    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.533199 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.533171    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f24bcba-cf62-4980-9875-af536f59a8d3-openshift-service-ca\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.533381 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.533228    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8mzr\" (UniqueName: \"kubernetes.io/projected/7f24bcba-cf62-4980-9875-af536f59a8d3-kube-api-access-h8mzr\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.533381 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.533258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-metrics-certs\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.533381 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.533292    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.533503 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:41:08.533484    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-serving-cert: secret "maas-api-e2e-explicit-b2dfe7-serving-cert" not found
Sep 10 14:41:08.533598 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:41:08.533584    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls podName:7f24bcba-cf62-4980-9875-af536f59a8d3 nodeName:}" failed. No retries permitted until 2026-09-10 14:41:09.033552452 +0000 UTC m=+1274.801275230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls") pod "maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p" (UID: "7f24bcba-cf62-4980-9875-af536f59a8d3") : secret "maas-api-e2e-explicit-b2dfe7-serving-cert" not found
Sep 10 14:41:08.533914 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.533888    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f24bcba-cf62-4980-9875-af536f59a8d3-openshift-service-ca\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.535981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.535960    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-metrics-certs\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.541540 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.541514    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8mzr\" (UniqueName: \"kubernetes.io/projected/7f24bcba-cf62-4980-9875-af536f59a8d3-kube-api-access-h8mzr\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:08.621329 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.621242    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:08.930094 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:08.929993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:09.037929 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:09.037896    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:09.038784 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:09.038732    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:09.040417 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:09.040399    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls\") pod \"maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") " pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:09.239763 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:09.239663    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:09.522188 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:09.522160    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"]
Sep 10 14:41:09.524434 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:41:09.524408    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f24bcba_cf62_4980_9875_af536f59a8d3.slice/crio-937f3b4351d6d3da657a8c854c24d6cbf1be63c59282d6b5027f6671f8ac6aab WatchSource:0}: Error finding container 937f3b4351d6d3da657a8c854c24d6cbf1be63c59282d6b5027f6671f8ac6aab: Status 404 returned error can't find the container with id 937f3b4351d6d3da657a8c854c24d6cbf1be63c59282d6b5027f6671f8ac6aab
Sep 10 14:41:10.153872 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:10.153787    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p" event={"ID":"7f24bcba-cf62-4980-9875-af536f59a8d3","Type":"ContainerStarted","Data":"5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c"}
Sep 10 14:41:10.153872 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:10.153844    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p" event={"ID":"7f24bcba-cf62-4980-9875-af536f59a8d3","Type":"ContainerStarted","Data":"937f3b4351d6d3da657a8c854c24d6cbf1be63c59282d6b5027f6671f8ac6aab"}
Sep 10 14:41:10.154058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:10.153969    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:10.170361 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:10.170307    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p" podStartSLOduration=1.8072078999999999 podStartE2EDuration="2.170286975s" totalImagesPullingTime="363.079075ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:41:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:41:10.168984281 +0000 UTC m=+1275.936707064" watchObservedRunningTime="2026-09-10 14:41:10.170286975 +0000 UTC m=+1275.938009760"
Sep 10 14:41:11.813084 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:11.813047    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:16.165318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:16.165287    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:41:17.617307 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:17.617256    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:27.529351 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.529319    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"]
Sep 10 14:41:27.529793 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.529525    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4" podUID="f7afb576-cc8a-4bed-a2a4-7936bb92d23b" containerName="maas-api" containerID="cri-o://97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b" gracePeriod=30
Sep 10 14:41:27.570675 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.570552    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:27.671140 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.671087    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:27.778273 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.778240    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:41:27.911823 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.911790    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-metrics-certs\") pod \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") "
Sep 10 14:41:27.912000 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.911847    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-maas-api-tls\") pod \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") "
Sep 10 14:41:27.912000 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.911907    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlzr9\" (UniqueName: \"kubernetes.io/projected/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-kube-api-access-rlzr9\") pod \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") "
Sep 10 14:41:27.912000 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.911951    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-openshift-service-ca\") pod \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\" (UID: \"f7afb576-cc8a-4bed-a2a4-7936bb92d23b\") "
Sep 10 14:41:27.912461 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.912362    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f7afb576-cc8a-4bed-a2a4-7936bb92d23b" (UID: "f7afb576-cc8a-4bed-a2a4-7936bb92d23b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:41:27.914052 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.914024    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-kube-api-access-rlzr9" (OuterVolumeSpecName: "kube-api-access-rlzr9") pod "f7afb576-cc8a-4bed-a2a4-7936bb92d23b" (UID: "f7afb576-cc8a-4bed-a2a4-7936bb92d23b"). InnerVolumeSpecName "kube-api-access-rlzr9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:41:27.914052 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.914029    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f7afb576-cc8a-4bed-a2a4-7936bb92d23b" (UID: "f7afb576-cc8a-4bed-a2a4-7936bb92d23b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:27.914210 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:27.914059    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "f7afb576-cc8a-4bed-a2a4-7936bb92d23b" (UID: "f7afb576-cc8a-4bed-a2a4-7936bb92d23b"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:41:28.013546 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.013513    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:41:28.013546 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.013541    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:41:28.013546 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.013550    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:41:28.013802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.013560    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlzr9\" (UniqueName: \"kubernetes.io/projected/f7afb576-cc8a-4bed-a2a4-7936bb92d23b-kube-api-access-rlzr9\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:41:28.205585 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.205494    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:28.254352 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.254325    2578 generic.go:356] "Generic (PLEG): container finished" podID="f7afb576-cc8a-4bed-a2a4-7936bb92d23b" containerID="97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b" exitCode=0
Sep 10 14:41:28.254550 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.254370    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"
Sep 10 14:41:28.254550 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.254402    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4" event={"ID":"f7afb576-cc8a-4bed-a2a4-7936bb92d23b","Type":"ContainerDied","Data":"97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b"}
Sep 10 14:41:28.254550 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.254444    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4" event={"ID":"f7afb576-cc8a-4bed-a2a4-7936bb92d23b","Type":"ContainerDied","Data":"5d51316e51283c0a95d54a6b076ae7a73cbd7fcc3e836b1916b4af23e28fe0bb"}
Sep 10 14:41:28.254550 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.254465    2578 scope.go:117] "RemoveContainer" containerID="97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b"
Sep 10 14:41:28.266302 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.266279    2578 scope.go:117] "RemoveContainer" containerID="97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b"
Sep 10 14:41:28.266759 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:41:28.266725    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b\": container with ID starting with 97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b not found: ID does not exist" containerID="97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b"
Sep 10 14:41:28.266863 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.266774    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b"} err="failed to get container status \"97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b\": rpc error: code = NotFound desc = could not find container \"97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b\": container with ID starting with 97b2dca7f7e91cc82089858288b869a20c1a2941d50ee396fac505d1b671e39b not found: ID does not exist"
Sep 10 14:41:28.277773 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.277727    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"]
Sep 10 14:41:28.279633 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.279609    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-b46e5fd4-8456646cb7-v7rs4"]
Sep 10 14:41:28.280437 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.280412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:28.601463 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.601429    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:28.700715 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.700687    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:28.724516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.724487    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7afb576-cc8a-4bed-a2a4-7936bb92d23b" path="/var/lib/kubelet/pods/f7afb576-cc8a-4bed-a2a4-7936bb92d23b/volumes"
Sep 10 14:41:28.800440 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:28.800401    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:29.386227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:29.386175    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:30.101625 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:30.101587    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:32.044201 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:32.044166    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:33.776473 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:33.776439    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:35.811111 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:35.811052    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:35.919591 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:35.919560    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:36.031289 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:36.031250    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:36.140062 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:36.140025    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:36.386189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:36.386156    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:36.523219 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:36.523132    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:41:57.915500 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:41:57.915460    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:00.828815 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:00.828785    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:01.027279 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:01.027245    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:02.379317 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:02.379280    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:08.558281 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:08.558245    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:08.665167 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:08.665119    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:08.788813 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:08.788777    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:11.493430 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.493400    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"]
Sep 10 14:42:11.494153 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.494120    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p" podUID="7f24bcba-cf62-4980-9875-af536f59a8d3" containerName="maas-api" containerID="cri-o://5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c" gracePeriod=30
Sep 10 14:42:11.521021 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.520994    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:11.594888 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.594845    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:11.693908 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.693639    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:11.761694 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.761649    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:11.787833 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.787810    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:42:11.905296 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.905265    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f24bcba-cf62-4980-9875-af536f59a8d3-openshift-service-ca\") pod \"7f24bcba-cf62-4980-9875-af536f59a8d3\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") "
Sep 10 14:42:11.905434 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.905343    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-metrics-certs\") pod \"7f24bcba-cf62-4980-9875-af536f59a8d3\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") "
Sep 10 14:42:11.905434 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.905367    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls\") pod \"7f24bcba-cf62-4980-9875-af536f59a8d3\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") "
Sep 10 14:42:11.905434 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.905386    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h8mzr\" (UniqueName: \"kubernetes.io/projected/7f24bcba-cf62-4980-9875-af536f59a8d3-kube-api-access-h8mzr\") pod \"7f24bcba-cf62-4980-9875-af536f59a8d3\" (UID: \"7f24bcba-cf62-4980-9875-af536f59a8d3\") "
Sep 10 14:42:11.905673 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.905635    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7f24bcba-cf62-4980-9875-af536f59a8d3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7f24bcba-cf62-4980-9875-af536f59a8d3" (UID: "7f24bcba-cf62-4980-9875-af536f59a8d3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:42:11.907632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.907602    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7f24bcba-cf62-4980-9875-af536f59a8d3" (UID: "7f24bcba-cf62-4980-9875-af536f59a8d3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:11.907632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.907629    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7f24bcba-cf62-4980-9875-af536f59a8d3-kube-api-access-h8mzr" (OuterVolumeSpecName: "kube-api-access-h8mzr") pod "7f24bcba-cf62-4980-9875-af536f59a8d3" (UID: "7f24bcba-cf62-4980-9875-af536f59a8d3"). InnerVolumeSpecName "kube-api-access-h8mzr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:42:11.907815 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:11.907686    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7f24bcba-cf62-4980-9875-af536f59a8d3" (UID: "7f24bcba-cf62-4980-9875-af536f59a8d3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:12.006865 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.006832    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7f24bcba-cf62-4980-9875-af536f59a8d3-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:42:12.006865 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.006859    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:42:12.006865 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.006869    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7f24bcba-cf62-4980-9875-af536f59a8d3-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:42:12.007072 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.006877    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8mzr\" (UniqueName: \"kubernetes.io/projected/7f24bcba-cf62-4980-9875-af536f59a8d3-kube-api-access-h8mzr\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:42:12.066662 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.066621    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:12.491569 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.491486    2578 generic.go:356] "Generic (PLEG): container finished" podID="7f24bcba-cf62-4980-9875-af536f59a8d3" containerID="5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c" exitCode=0
Sep 10 14:42:12.491569 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.491526    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"
Sep 10 14:42:12.491797 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.491583    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p" event={"ID":"7f24bcba-cf62-4980-9875-af536f59a8d3","Type":"ContainerDied","Data":"5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c"}
Sep 10 14:42:12.491797 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.491633    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p" event={"ID":"7f24bcba-cf62-4980-9875-af536f59a8d3","Type":"ContainerDied","Data":"937f3b4351d6d3da657a8c854c24d6cbf1be63c59282d6b5027f6671f8ac6aab"}
Sep 10 14:42:12.491797 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.491655    2578 scope.go:117] "RemoveContainer" containerID="5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c"
Sep 10 14:42:12.501315 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.501128    2578 scope.go:117] "RemoveContainer" containerID="5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c"
Sep 10 14:42:12.501526 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:42:12.501343    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c\": container with ID starting with 5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c not found: ID does not exist" containerID="5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c"
Sep 10 14:42:12.501526 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.501364    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c"} err="failed to get container status \"5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c\": rpc error: code = NotFound desc = could not find container \"5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c\": container with ID starting with 5b10773acf3babc26998e6d6c63980ece7e40e8d8e0a0b5ebcdf65d7fe06121c not found: ID does not exist"
Sep 10 14:42:12.511784 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.511759    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"]
Sep 10 14:42:12.518077 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.518058    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-explicit-b2dfe7-65fd7f85d-xgc7p"]
Sep 10 14:42:12.724674 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:12.724644    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7f24bcba-cf62-4980-9875-af536f59a8d3" path="/var/lib/kubelet/pods/7f24bcba-cf62-4980-9875-af536f59a8d3/volumes"
Sep 10 14:42:18.098249 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:18.098163    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:18.198376 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:18.198336    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:18.347533 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:18.347497    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:18.794307 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:18.794274    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:19.086844 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:19.086805    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:19.331069 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:19.331037    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:20.046659 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:20.046621    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:20.142770 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:20.142715    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:20.243080 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:20.243046    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:20.936232 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:20.936204    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:21.038430 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:21.038385    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:24.559872 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:24.559831    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:30.248196 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:30.248156    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:30.353590 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:30.353540    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:30.449020 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:30.448978    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:32.345606 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:32.345568    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:38.191932 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:38.191881    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:38.305304 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:38.305268    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:38.513719 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:38.513632    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:38.625896 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:38.625865    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:40.951502 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:40.951466    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:41.306041 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:41.305951    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:41.745532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:41.745491    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:42.270835 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:42.270784    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:42.484807 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:42.484741    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:43.644688 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:43.644651    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:47.822495 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:47.822462    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:47.926573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:47.926534    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:48.027426 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:48.027395    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:48.202653 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:48.202568    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:48.789685 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:48.789647    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:48.932563 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:48.932526    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:49.027825 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:49.027784    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:50.457841 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:50.457809    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:52.903456 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:52.903414    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"]
Sep 10 14:42:52.903951 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:52.903702    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t" podUID="1c801c14-8727-4b1a-9de4-757b247b7c21" containerName="maas-api" containerID="cri-o://eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87" gracePeriod=30
Sep 10 14:42:52.936169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:52.936140    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:53.029631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.029597    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:53.129146 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.129114    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:53.155955 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.155901    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:42:53.176129 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.173952    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c801c14-8727-4b1a-9de4-757b247b7c21-openshift-service-ca\") pod \"1c801c14-8727-4b1a-9de4-757b247b7c21\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") "
Sep 10 14:42:53.176129 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.174031    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls\") pod \"1c801c14-8727-4b1a-9de4-757b247b7c21\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") "
Sep 10 14:42:53.176129 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.174082    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-metrics-certs\") pod \"1c801c14-8727-4b1a-9de4-757b247b7c21\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") "
Sep 10 14:42:53.176129 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.174119    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mf4jp\" (UniqueName: \"kubernetes.io/projected/1c801c14-8727-4b1a-9de4-757b247b7c21-kube-api-access-mf4jp\") pod \"1c801c14-8727-4b1a-9de4-757b247b7c21\" (UID: \"1c801c14-8727-4b1a-9de4-757b247b7c21\") "
Sep 10 14:42:53.176129 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.175108    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1c801c14-8727-4b1a-9de4-757b247b7c21-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1c801c14-8727-4b1a-9de4-757b247b7c21" (UID: "1c801c14-8727-4b1a-9de4-757b247b7c21"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:42:53.177511 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.177422    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1c801c14-8727-4b1a-9de4-757b247b7c21-kube-api-access-mf4jp" (OuterVolumeSpecName: "kube-api-access-mf4jp") pod "1c801c14-8727-4b1a-9de4-757b247b7c21" (UID: "1c801c14-8727-4b1a-9de4-757b247b7c21"). InnerVolumeSpecName "kube-api-access-mf4jp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:42:53.178933 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.178913    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1c801c14-8727-4b1a-9de4-757b247b7c21" (UID: "1c801c14-8727-4b1a-9de4-757b247b7c21"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:53.179094 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.179067    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "1c801c14-8727-4b1a-9de4-757b247b7c21" (UID: "1c801c14-8727-4b1a-9de4-757b247b7c21"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:42:53.230151 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.230119    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:53.275171 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.275142    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mf4jp\" (UniqueName: \"kubernetes.io/projected/1c801c14-8727-4b1a-9de4-757b247b7c21-kube-api-access-mf4jp\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:42:53.275171 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.275172    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1c801c14-8727-4b1a-9de4-757b247b7c21-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:42:53.275354 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.275184    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:42:53.275354 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.275193    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1c801c14-8727-4b1a-9de4-757b247b7c21-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:42:53.517224 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.517143    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:53.700338 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.700306    2578 generic.go:356] "Generic (PLEG): container finished" podID="1c801c14-8727-4b1a-9de4-757b247b7c21" containerID="eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87" exitCode=0
Sep 10 14:42:53.700509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.700350    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"
Sep 10 14:42:53.700509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.700351    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t" event={"ID":"1c801c14-8727-4b1a-9de4-757b247b7c21","Type":"ContainerDied","Data":"eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87"}
Sep 10 14:42:53.700509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.700489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t" event={"ID":"1c801c14-8727-4b1a-9de4-757b247b7c21","Type":"ContainerDied","Data":"d2502471ace3443aa4ded11524b33c1cd87d6f31b3df39ec07057b6166ae241d"}
Sep 10 14:42:53.700632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.700518    2578 scope.go:117] "RemoveContainer" containerID="eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87"
Sep 10 14:42:53.709796 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.709781    2578 scope.go:117] "RemoveContainer" containerID="eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87"
Sep 10 14:42:53.710083 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:42:53.710064    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87\": container with ID starting with eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87 not found: ID does not exist" containerID="eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87"
Sep 10 14:42:53.710124 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.710096    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87"} err="failed to get container status \"eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87\": rpc error: code = NotFound desc = could not find container \"eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87\": container with ID starting with eed29323db914163d451379febb41a84458ef54f0033a55c835b90741a3c8b87 not found: ID does not exist"
Sep 10 14:42:53.722465 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.722440    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"]
Sep 10 14:42:53.726189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:53.726168    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-c892fe-7b588bd4c6-zmt2t"]
Sep 10 14:42:54.723350 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:54.723319    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1c801c14-8727-4b1a-9de4-757b247b7c21" path="/var/lib/kubelet/pods/1c801c14-8727-4b1a-9de4-757b247b7c21/volumes"
Sep 10 14:42:56.223855 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:56.223814    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:56.325568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:56.325530    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:56.852967 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:56.852933    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:59.533104 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:59.533069    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:59.611326 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:59.611285    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:42:59.781499 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:42:59.781445    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:00.224372 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:00.224331    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:00.517073 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:00.516993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:02.842801 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:02.842763    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:02.952976 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:02.952940    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:03.044606 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:03.044567    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:03.328669 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:03.328629    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:03.823261 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:03.823220    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:06.002038 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:06.002008    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:06.100853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:06.100805    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:07.191737 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:07.191703    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:07.297690 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:07.297654    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:07.399644 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:07.399599    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:07.745610 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:07.745573    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:07.860611 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:07.860572    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:10.238674 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:10.238642    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:11.044410 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.044380    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"]
Sep 10 14:43:11.044618 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.044591    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd" podUID="04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" containerName="maas-api" containerID="cri-o://919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14" gracePeriod=30
Sep 10 14:43:11.074014 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.073972    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:11.113684 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.113656    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"]
Sep 10 14:43:11.113915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.113891    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7" podUID="80010163-72d3-4558-9a49-06822081e1b9" containerName="payload-processing" containerID="cri-o://7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79" gracePeriod=30
Sep 10 14:43:11.172620 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.172582    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:11.271913 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.271834    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:11.372388 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.372342    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:11.379002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.378979    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:43:11.403024 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.402997    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:43:11.436429 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.436402    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-openshift-service-ca\") pod \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") "
Sep 10 14:43:11.436581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.436498    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whjrf\" (UniqueName: \"kubernetes.io/projected/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-kube-api-access-whjrf\") pod \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") "
Sep 10 14:43:11.436581 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.436555    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-metrics-certs\") pod \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") "
Sep 10 14:43:11.436699 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.436623    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls\") pod \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\" (UID: \"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed\") "
Sep 10 14:43:11.436776 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.436738    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" (UID: "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:11.436937 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.436903    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:43:11.438691 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.438657    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" (UID: "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:11.438810 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.438719    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-kube-api-access-whjrf" (OuterVolumeSpecName: "kube-api-access-whjrf") pod "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" (UID: "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed"). InnerVolumeSpecName "kube-api-access-whjrf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:11.438850 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.438837    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" (UID: "04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:43:11.537511 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.537483    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klvsh\" (UniqueName: \"kubernetes.io/projected/80010163-72d3-4558-9a49-06822081e1b9-kube-api-access-klvsh\") pod \"80010163-72d3-4558-9a49-06822081e1b9\" (UID: \"80010163-72d3-4558-9a49-06822081e1b9\") "
Sep 10 14:43:11.537702 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.537527    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80010163-72d3-4558-9a49-06822081e1b9-plugins-config-volume\") pod \"80010163-72d3-4558-9a49-06822081e1b9\" (UID: \"80010163-72d3-4558-9a49-06822081e1b9\") "
Sep 10 14:43:11.537828 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.537815    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:43:11.537907 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.537831    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:43:11.537907 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.537841    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whjrf\" (UniqueName: \"kubernetes.io/projected/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed-kube-api-access-whjrf\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:43:11.538003 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.537949    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/80010163-72d3-4558-9a49-06822081e1b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "80010163-72d3-4558-9a49-06822081e1b9" (UID: "80010163-72d3-4558-9a49-06822081e1b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:43:11.539624 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.539600    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/80010163-72d3-4558-9a49-06822081e1b9-kube-api-access-klvsh" (OuterVolumeSpecName: "kube-api-access-klvsh") pod "80010163-72d3-4558-9a49-06822081e1b9" (UID: "80010163-72d3-4558-9a49-06822081e1b9"). InnerVolumeSpecName "kube-api-access-klvsh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:43:11.638301 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.638271    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klvsh\" (UniqueName: \"kubernetes.io/projected/80010163-72d3-4558-9a49-06822081e1b9-kube-api-access-klvsh\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:43:11.638431 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.638297    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80010163-72d3-4558-9a49-06822081e1b9-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:43:11.662106 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.662072    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:11.795125 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.795088    2578 generic.go:356] "Generic (PLEG): container finished" podID="80010163-72d3-4558-9a49-06822081e1b9" containerID="7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79" exitCode=0
Sep 10 14:43:11.795328 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.795133    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"
Sep 10 14:43:11.795328 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.795182    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7" event={"ID":"80010163-72d3-4558-9a49-06822081e1b9","Type":"ContainerDied","Data":"7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79"}
Sep 10 14:43:11.795328 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.795232    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7" event={"ID":"80010163-72d3-4558-9a49-06822081e1b9","Type":"ContainerDied","Data":"f9190425b3d66cb8733c627bb7f62bc7913f26957b6533b0e8f289979d64dc71"}
Sep 10 14:43:11.795328 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.795262    2578 scope.go:117] "RemoveContainer" containerID="7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79"
Sep 10 14:43:11.796802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.796776    2578 generic.go:356] "Generic (PLEG): container finished" podID="04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" containerID="919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14" exitCode=0
Sep 10 14:43:11.796906 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.796866    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd" event={"ID":"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed","Type":"ContainerDied","Data":"919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14"}
Sep 10 14:43:11.796955 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.796917    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd" event={"ID":"04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed","Type":"ContainerDied","Data":"1bfd02c89c235393ae9815a40ab7f1b1af2825b4cf2dd970b2d69099ab8fc872"}
Sep 10 14:43:11.796955 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.796922    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"
Sep 10 14:43:11.806350 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.806329    2578 scope.go:117] "RemoveContainer" containerID="7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79"
Sep 10 14:43:11.806627 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:43:11.806602    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79\": container with ID starting with 7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79 not found: ID does not exist" containerID="7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79"
Sep 10 14:43:11.806771 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.806634    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79"} err="failed to get container status \"7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79\": rpc error: code = NotFound desc = could not find container \"7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79\": container with ID starting with 7404e35448b4c47b223d266dfa7ff81c2a020d2ee521ae8d88178849d5cfce79 not found: ID does not exist"
Sep 10 14:43:11.806771 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.806652    2578 scope.go:117] "RemoveContainer" containerID="919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14"
Sep 10 14:43:11.814869 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.814846    2578 scope.go:117] "RemoveContainer" containerID="919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14"
Sep 10 14:43:11.815152 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:43:11.815136    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14\": container with ID starting with 919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14 not found: ID does not exist" containerID="919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14"
Sep 10 14:43:11.815213 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.815159    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14"} err="failed to get container status \"919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14\": rpc error: code = NotFound desc = could not find container \"919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14\": container with ID starting with 919907edb6639ecf6f72c1e6fc2d11c6278b4d8de54797549c91ea673bf34f14 not found: ID does not exist"
Sep 10 14:43:11.816141 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.816124    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"]
Sep 10 14:43:11.821806 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.820713    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-2365fe-bfc98b7bf-rgth7"]
Sep 10 14:43:11.833457 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.833431    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"]
Sep 10 14:43:11.842035 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:11.842016    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-2365fe-55cdd85d5f-ch5kd"]
Sep 10 14:43:12.723717 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:12.723685    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" path="/var/lib/kubelet/pods/04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed/volumes"
Sep 10 14:43:12.724098 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:12.724077    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="80010163-72d3-4558-9a49-06822081e1b9" path="/var/lib/kubelet/pods/80010163-72d3-4558-9a49-06822081e1b9/volumes"
Sep 10 14:43:17.749276 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:17.749239    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:17.917741 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:17.917698    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:18.039460 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:18.039367    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:18.555481 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:18.555442    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:18.880600 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:18.880559    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:19.017469 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:19.017427    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:20.402659 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:20.402625    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:21.965394 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:21.965361    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:22.063682 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:22.063647    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:23.079122 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079087    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"]
Sep 10 14:43:23.079683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079664    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1c801c14-8727-4b1a-9de4-757b247b7c21" containerName="maas-api"
Sep 10 14:43:23.079735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079687    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1c801c14-8727-4b1a-9de4-757b247b7c21" containerName="maas-api"
Sep 10 14:43:23.079735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079705    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="80010163-72d3-4558-9a49-06822081e1b9" containerName="payload-processing"
Sep 10 14:43:23.079735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079714    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="80010163-72d3-4558-9a49-06822081e1b9" containerName="payload-processing"
Sep 10 14:43:23.079735 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079734    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" containerName="maas-api"
Sep 10 14:43:23.079889 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079760    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" containerName="maas-api"
Sep 10 14:43:23.079889 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079784    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7afb576-cc8a-4bed-a2a4-7936bb92d23b" containerName="maas-api"
Sep 10 14:43:23.079889 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079794    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7afb576-cc8a-4bed-a2a4-7936bb92d23b" containerName="maas-api"
Sep 10 14:43:23.079889 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079814    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7f24bcba-cf62-4980-9875-af536f59a8d3" containerName="maas-api"
Sep 10 14:43:23.079889 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079825    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7f24bcba-cf62-4980-9875-af536f59a8d3" containerName="maas-api"
Sep 10 14:43:23.080058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079918    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1c801c14-8727-4b1a-9de4-757b247b7c21" containerName="maas-api"
Sep 10 14:43:23.080058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079932    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7afb576-cc8a-4bed-a2a4-7936bb92d23b" containerName="maas-api"
Sep 10 14:43:23.080058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079939    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7f24bcba-cf62-4980-9875-af536f59a8d3" containerName="maas-api"
Sep 10 14:43:23.080058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079945    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="04bc9d14-100f-42e2-a0fc-d0b6fdfd0eed" containerName="maas-api"
Sep 10 14:43:23.080058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.079950    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="80010163-72d3-4558-9a49-06822081e1b9" containerName="payload-processing"
Sep 10 14:43:23.083042 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.083022    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.085068 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.085038    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w3-85d86b-serving-cert\""
Sep 10 14:43:23.092633 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.092610    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"]
Sep 10 14:43:23.118766 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.118721    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:23.216841 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.216811    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:23.244212 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.244183    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wddqd\" (UniqueName: \"kubernetes.io/projected/b5907479-c0bb-4952-8e9e-a8bc23331d79-kube-api-access-wddqd\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.244389 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.244243    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.244389 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.244326    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5907479-c0bb-4952-8e9e-a8bc23331d79-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.244389 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.244371    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.299808 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.299775    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:23.345614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.345538    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wddqd\" (UniqueName: \"kubernetes.io/projected/b5907479-c0bb-4952-8e9e-a8bc23331d79-kube-api-access-wddqd\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.345614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.345598    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.345854 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.345785    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5907479-c0bb-4952-8e9e-a8bc23331d79-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.345854 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.345835    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.346380 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.346349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5907479-c0bb-4952-8e9e-a8bc23331d79-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.348042 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.348021    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-maas-api-tls\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.348139 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.348122    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-metrics-certs\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.352935 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.352917    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wddqd\" (UniqueName: \"kubernetes.io/projected/b5907479-c0bb-4952-8e9e-a8bc23331d79-kube-api-access-wddqd\") pod \"maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.395600 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.395564    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:23.672870 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.672779    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"]
Sep 10 14:43:23.677049 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:43:23.677014    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5907479_c0bb_4952_8e9e_a8bc23331d79.slice/crio-896c0c735022febf0926dfd6dca3d6ffbe9aeec5aca1e5d0c9ad0086db57b0e7 WatchSource:0}: Error finding container 896c0c735022febf0926dfd6dca3d6ffbe9aeec5aca1e5d0c9ad0086db57b0e7: Status 404 returned error can't find the container with id 896c0c735022febf0926dfd6dca3d6ffbe9aeec5aca1e5d0c9ad0086db57b0e7
Sep 10 14:43:23.678523 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.678490    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:43:23.695582 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.695541    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:23.798500 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.798461    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:23.811236 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.811210    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"]
Sep 10 14:43:23.815954 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.815937    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:23.817880 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.817862    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-85d86b-dockercfg-2cr6r\""
Sep 10 14:43:23.817960 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.817863    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-85d86b\""
Sep 10 14:43:23.823892 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.823682    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"]
Sep 10 14:43:23.827183 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.827153    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:23.828882 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.828860    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"]
Sep 10 14:43:23.838099 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.838075    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"]
Sep 10 14:43:23.862307 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.862276    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh" event={"ID":"b5907479-c0bb-4952-8e9e-a8bc23331d79","Type":"ContainerStarted","Data":"896c0c735022febf0926dfd6dca3d6ffbe9aeec5aca1e5d0c9ad0086db57b0e7"}
Sep 10 14:43:23.952286 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.952213    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1da6868-fed8-4751-8e1c-d2444892407a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk\" (UID: \"c1da6868-fed8-4751-8e1c-d2444892407a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:23.952286 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.952278    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ndmk\" (UniqueName: \"kubernetes.io/projected/c1da6868-fed8-4751-8e1c-d2444892407a-kube-api-access-7ndmk\") pod \"payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk\" (UID: \"c1da6868-fed8-4751-8e1c-d2444892407a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:23.952483 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.952370    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7\" (UID: \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:23.952483 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:23.952395    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwrp4\" (UniqueName: \"kubernetes.io/projected/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-kube-api-access-vwrp4\") pod \"payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7\" (UID: \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:24.053198 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.053169    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7ndmk\" (UniqueName: \"kubernetes.io/projected/c1da6868-fed8-4751-8e1c-d2444892407a-kube-api-access-7ndmk\") pod \"payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk\" (UID: \"c1da6868-fed8-4751-8e1c-d2444892407a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:24.053285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.053238    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7\" (UID: \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:24.053285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.053265    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwrp4\" (UniqueName: \"kubernetes.io/projected/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-kube-api-access-vwrp4\") pod \"payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7\" (UID: \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:24.053371 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.053326    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1da6868-fed8-4751-8e1c-d2444892407a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk\" (UID: \"c1da6868-fed8-4751-8e1c-d2444892407a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:24.054168 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.054132    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7\" (UID: \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:24.054282 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.054205    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1da6868-fed8-4751-8e1c-d2444892407a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk\" (UID: \"c1da6868-fed8-4751-8e1c-d2444892407a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:24.064464 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.064389    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwrp4\" (UniqueName: \"kubernetes.io/projected/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-kube-api-access-vwrp4\") pod \"payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7\" (UID: \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:24.064464 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.064401    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ndmk\" (UniqueName: \"kubernetes.io/projected/c1da6868-fed8-4751-8e1c-d2444892407a-kube-api-access-7ndmk\") pod \"payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk\" (UID: \"c1da6868-fed8-4751-8e1c-d2444892407a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:24.127875 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.127845    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:24.138528 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.138509    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:24.405728 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.405368    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"]
Sep 10 14:43:24.408341 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:43:24.408308    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1da6868_fed8_4751_8e1c_d2444892407a.slice/crio-c739886f7725a424be42db3378167e3414e9e412167a0c8b609bcf29e4f2db59 WatchSource:0}: Error finding container c739886f7725a424be42db3378167e3414e9e412167a0c8b609bcf29e4f2db59: Status 404 returned error can't find the container with id c739886f7725a424be42db3378167e3414e9e412167a0c8b609bcf29e4f2db59
Sep 10 14:43:24.418707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.418686    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"]
Sep 10 14:43:24.421870 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:43:24.421843    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59f74e3b_24b1_49d0_b3b4_110c75cfc65d.slice/crio-78a19ae0d1f1d6ccac81e04a09ad3783a5a90e953de14f91ddc28f915cfca21e WatchSource:0}: Error finding container 78a19ae0d1f1d6ccac81e04a09ad3783a5a90e953de14f91ddc28f915cfca21e: Status 404 returned error can't find the container with id 78a19ae0d1f1d6ccac81e04a09ad3783a5a90e953de14f91ddc28f915cfca21e
Sep 10 14:43:24.873844 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.873798    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh" event={"ID":"b5907479-c0bb-4952-8e9e-a8bc23331d79","Type":"ContainerStarted","Data":"652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2"}
Sep 10 14:43:24.874033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.873880    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:24.875627 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.875594    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk" event={"ID":"c1da6868-fed8-4751-8e1c-d2444892407a","Type":"ContainerStarted","Data":"da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191"}
Sep 10 14:43:24.875796 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.875651    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk" event={"ID":"c1da6868-fed8-4751-8e1c-d2444892407a","Type":"ContainerStarted","Data":"c739886f7725a424be42db3378167e3414e9e412167a0c8b609bcf29e4f2db59"}
Sep 10 14:43:24.875876 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.875855    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:24.877456 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.877433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7" event={"ID":"59f74e3b-24b1-49d0-b3b4-110c75cfc65d","Type":"ContainerStarted","Data":"0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8"}
Sep 10 14:43:24.877571 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.877474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7" event={"ID":"59f74e3b-24b1-49d0-b3b4-110c75cfc65d","Type":"ContainerStarted","Data":"78a19ae0d1f1d6ccac81e04a09ad3783a5a90e953de14f91ddc28f915cfca21e"}
Sep 10 14:43:24.877571 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.877498    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:24.891337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.891270    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh" podStartSLOduration=1.523257933 podStartE2EDuration="1.891254777s" totalImagesPullingTime="367.996844ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:24.889180084 +0000 UTC m=+1410.656902864" watchObservedRunningTime="2026-09-10 14:43:24.891254777 +0000 UTC m=+1410.658977556"
Sep 10 14:43:24.904454 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.904412    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7" podStartSLOduration=1.904398212 podStartE2EDuration="1.904398212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:24.902596369 +0000 UTC m=+1410.670319151" watchObservedRunningTime="2026-09-10 14:43:24.904398212 +0000 UTC m=+1410.672120992"
Sep 10 14:43:24.918453 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:24.918410    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk" podStartSLOduration=1.91839847 podStartE2EDuration="1.91839847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:43:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:43:24.916280741 +0000 UTC m=+1410.684003512" watchObservedRunningTime="2026-09-10 14:43:24.91839847 +0000 UTC m=+1410.686121249"
Sep 10 14:43:28.201780 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:28.201726    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:30.889979 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:30.889950    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:43:30.920288 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:30.920258    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:31.567142 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:31.567111    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:31.666140 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:31.666108    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:32.011831 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:32.011717    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:35.886684 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:35.886642    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:43:35.887087 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:35.886840    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:43:38.066825 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:38.066788    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:38.156610 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:38.156580    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:44.085868 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:44.085823    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:46.910874 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:46.910793    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:47.293082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:47.293002    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:47.550198 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:47.550115    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:47.887114 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:47.887081    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:48.656536 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:48.656504    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:49.332706 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:49.332673    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:49.406107 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:49.406076    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:49.507893 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:49.507860    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:49.577853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:49.577822    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:49.908642 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:49.908613    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:50.212053 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:50.211963    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:50.311511 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:50.311484    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:51.377809 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:51.377773    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:51.551566 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:51.551523    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:51.647620 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:51.647532    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:51.986448 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:51.986369    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:52.094024 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:52.093984    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:56.060178 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:56.060138    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:56.164966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:56.164931    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:56.346910 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:56.346871    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:56.674576 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:56.674488    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:56.766157 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:56.766126    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:43:56.929609 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:43:56.929515    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:03.066187 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:03.066147    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:03.174979 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:03.174942    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:03.267147 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:03.267090    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:03.392808 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:03.392701    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:09.701706 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:09.701668    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:09.804325 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:09.804295    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:09.902097 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:09.902064    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:10.120125 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:10.120090    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:10.807779 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:10.807722    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:13.534656 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:13.534622    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:13.634321 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:13.634285    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:14.706966 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:14.706937    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:14.805134 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:14.805100    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:14.996038 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:14.995941    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:15.094090 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.094054    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:15.265250 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.265164    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:15.363149 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.363116    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:15.368256 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.368225    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"]
Sep 10 14:44:15.373440 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.373420    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:15.375458 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.375433    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-bc9af3-dockercfg-2h4vg\""
Sep 10 14:44:15.375578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.375443    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-bc9af3\""
Sep 10 14:44:15.382352 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.382115    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"]
Sep 10 14:44:15.515675 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.515593    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8718a975-1b60-40c8-96bf-0890ba8e50a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8\" (UID: \"8718a975-1b60-40c8-96bf-0890ba8e50a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:15.515888 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.515808    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fnqln\" (UniqueName: \"kubernetes.io/projected/8718a975-1b60-40c8-96bf-0890ba8e50a8-kube-api-access-fnqln\") pod \"payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8\" (UID: \"8718a975-1b60-40c8-96bf-0890ba8e50a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:15.616303 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.616264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8718a975-1b60-40c8-96bf-0890ba8e50a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8\" (UID: \"8718a975-1b60-40c8-96bf-0890ba8e50a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:15.616501 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.616353    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fnqln\" (UniqueName: \"kubernetes.io/projected/8718a975-1b60-40c8-96bf-0890ba8e50a8-kube-api-access-fnqln\") pod \"payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8\" (UID: \"8718a975-1b60-40c8-96bf-0890ba8e50a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:15.617061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.617034    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8718a975-1b60-40c8-96bf-0890ba8e50a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8\" (UID: \"8718a975-1b60-40c8-96bf-0890ba8e50a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:15.623632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.623608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fnqln\" (UniqueName: \"kubernetes.io/projected/8718a975-1b60-40c8-96bf-0890ba8e50a8-kube-api-access-fnqln\") pod \"payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8\" (UID: \"8718a975-1b60-40c8-96bf-0890ba8e50a8\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:15.686447 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.686414    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:15.958017 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:15.957989    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"]
Sep 10 14:44:15.960077 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:44:15.960049    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8718a975_1b60_40c8_96bf_0890ba8e50a8.slice/crio-86712ad9ac4355c1b364765a929d44421382576616298e56624be6a18ee47462 WatchSource:0}: Error finding container 86712ad9ac4355c1b364765a929d44421382576616298e56624be6a18ee47462: Status 404 returned error can't find the container with id 86712ad9ac4355c1b364765a929d44421382576616298e56624be6a18ee47462
Sep 10 14:44:16.138267 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:16.138229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8" event={"ID":"8718a975-1b60-40c8-96bf-0890ba8e50a8","Type":"ContainerStarted","Data":"076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a"}
Sep 10 14:44:16.138433 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:16.138290    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8" event={"ID":"8718a975-1b60-40c8-96bf-0890ba8e50a8","Type":"ContainerStarted","Data":"86712ad9ac4355c1b364765a929d44421382576616298e56624be6a18ee47462"}
Sep 10 14:44:16.138433 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:16.138365    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:16.153932 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:16.153880    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8" podStartSLOduration=1.153864354 podStartE2EDuration="1.153864354s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:44:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:44:16.152243613 +0000 UTC m=+1461.919966392" watchObservedRunningTime="2026-09-10 14:44:16.153864354 +0000 UTC m=+1461.921587134"
Sep 10 14:44:17.604594 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:17.604562    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:17.705653 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:17.705619    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:27.145882 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:27.145850    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:44:35.018963 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:35.018927    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:38.257391 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:38.257355    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:38.613842 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:38.613809    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:38.915082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:38.914993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:39.132425 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:39.132389    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:41.488672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:41.488642    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:41.584707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:41.584678    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:42.632158 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.632123    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"]
Sep 10 14:44:42.635964 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.635944    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.637736 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.637714    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-86a2c6-serving-cert\""
Sep 10 14:44:42.645926 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.645904    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"]
Sep 10 14:44:42.731013 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.730976    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:42.753907 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.753880    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46b9fc5e-2632-4e29-a87a-8c30a505fe76-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.754071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.753970    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.754071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.754013    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.754166 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.754117    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fdjd\" (UniqueName: \"kubernetes.io/projected/46b9fc5e-2632-4e29-a87a-8c30a505fe76-kube-api-access-7fdjd\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.832015 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.831977    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:42.855136 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.855109    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fdjd\" (UniqueName: \"kubernetes.io/projected/46b9fc5e-2632-4e29-a87a-8c30a505fe76-kube-api-access-7fdjd\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.855311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.855140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46b9fc5e-2632-4e29-a87a-8c30a505fe76-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.855311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.855180    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.855311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.855211    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.855475 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:44:42.855321    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-serving-cert: secret "maas-api-e2e-shared-b-w2-86a2c6-serving-cert" not found
Sep 10 14:44:42.855475 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:44:42.855382    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls podName:46b9fc5e-2632-4e29-a87a-8c30a505fe76 nodeName:}" failed. No retries permitted until 2026-09-10 14:44:43.355365302 +0000 UTC m=+1489.123088062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls") pod "maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg" (UID: "46b9fc5e-2632-4e29-a87a-8c30a505fe76") : secret "maas-api-e2e-shared-b-w2-86a2c6-serving-cert" not found
Sep 10 14:44:42.855723 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.855702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46b9fc5e-2632-4e29-a87a-8c30a505fe76-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.857683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.857662    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:42.863234 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:42.863214    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fdjd\" (UniqueName: \"kubernetes.io/projected/46b9fc5e-2632-4e29-a87a-8c30a505fe76-kube-api-access-7fdjd\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:43.195062 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.195025    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:43.312644 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.312605    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:43.359180 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.359147    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:43.359376 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:44:43.359313    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-serving-cert: secret "maas-api-e2e-shared-b-w2-86a2c6-serving-cert" not found
Sep 10 14:44:43.359446 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:44:43.359405    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls podName:46b9fc5e-2632-4e29-a87a-8c30a505fe76 nodeName:}" failed. No retries permitted until 2026-09-10 14:44:44.359382899 +0000 UTC m=+1490.127105854 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls") pod "maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg" (UID: "46b9fc5e-2632-4e29-a87a-8c30a505fe76") : secret "maas-api-e2e-shared-b-w2-86a2c6-serving-cert" not found
Sep 10 14:44:43.477067 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.476971    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:44:43.511139 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.511109    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"]
Sep 10 14:44:43.514886 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.514866    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:43.516773 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.516740    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-86a2c6\""
Sep 10 14:44:43.516955 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.516938    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-86a2c6-dockercfg-qjb9l\""
Sep 10 14:44:43.522785 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.522695    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"]
Sep 10 14:44:43.561738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.561713    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851ae358-b268-480f-9673-92f449cd705c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99\" (UID: \"851ae358-b268-480f-9673-92f449cd705c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:43.561908 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.561856    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rf8fx\" (UniqueName: \"kubernetes.io/projected/851ae358-b268-480f-9673-92f449cd705c-kube-api-access-rf8fx\") pod \"payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99\" (UID: \"851ae358-b268-480f-9673-92f449cd705c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:43.663047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.663013    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rf8fx\" (UniqueName: \"kubernetes.io/projected/851ae358-b268-480f-9673-92f449cd705c-kube-api-access-rf8fx\") pod \"payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99\" (UID: \"851ae358-b268-480f-9673-92f449cd705c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:43.663465 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.663124    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851ae358-b268-480f-9673-92f449cd705c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99\" (UID: \"851ae358-b268-480f-9673-92f449cd705c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:43.663827 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.663806    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851ae358-b268-480f-9673-92f449cd705c-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99\" (UID: \"851ae358-b268-480f-9673-92f449cd705c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:43.671933 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.671900    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rf8fx\" (UniqueName: \"kubernetes.io/projected/851ae358-b268-480f-9673-92f449cd705c-kube-api-access-rf8fx\") pod \"payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99\" (UID: \"851ae358-b268-480f-9673-92f449cd705c\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:43.826523 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:43.826497    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:44.108394 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.108370    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"]
Sep 10 14:44:44.110618 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:44:44.110584    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod851ae358_b268_480f_9673_92f449cd705c.slice/crio-16941dd47a5dbce5bc30349ce69852e6397ee2a82d5d65f2b3e9926d30e66c8c WatchSource:0}: Error finding container 16941dd47a5dbce5bc30349ce69852e6397ee2a82d5d65f2b3e9926d30e66c8c: Status 404 returned error can't find the container with id 16941dd47a5dbce5bc30349ce69852e6397ee2a82d5d65f2b3e9926d30e66c8c
Sep 10 14:44:44.281957 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.281913    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99" event={"ID":"851ae358-b268-480f-9673-92f449cd705c","Type":"ContainerStarted","Data":"5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c"}
Sep 10 14:44:44.282123 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.281980    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99" event={"ID":"851ae358-b268-480f-9673-92f449cd705c","Type":"ContainerStarted","Data":"16941dd47a5dbce5bc30349ce69852e6397ee2a82d5d65f2b3e9926d30e66c8c"}
Sep 10 14:44:44.282123 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.282030    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:44:44.298245 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.298203    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99" podStartSLOduration=1.2981892560000001 podStartE2EDuration="1.298189256s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:44:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:44:44.296739996 +0000 UTC m=+1490.064462775" watchObservedRunningTime="2026-09-10 14:44:44.298189256 +0000 UTC m=+1490.065912080"
Sep 10 14:44:44.371006 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.370937    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:44.373444 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.373417    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:44.446934 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.446901    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:44.726207 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:44.726181    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"]
Sep 10 14:44:44.727321 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:44:44.727297    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46b9fc5e_2632_4e29_a87a_8c30a505fe76.slice/crio-ee26b612f22ed929fa0e58704326eb6212835265e03f68451387c8314ce21f82 WatchSource:0}: Error finding container ee26b612f22ed929fa0e58704326eb6212835265e03f68451387c8314ce21f82: Status 404 returned error can't find the container with id ee26b612f22ed929fa0e58704326eb6212835265e03f68451387c8314ce21f82
Sep 10 14:44:45.290930 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:45.290882    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg" event={"ID":"46b9fc5e-2632-4e29-a87a-8c30a505fe76","Type":"ContainerStarted","Data":"0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e"}
Sep 10 14:44:45.291120 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:45.290949    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg" event={"ID":"46b9fc5e-2632-4e29-a87a-8c30a505fe76","Type":"ContainerStarted","Data":"ee26b612f22ed929fa0e58704326eb6212835265e03f68451387c8314ce21f82"}
Sep 10 14:44:45.291202 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:45.291127    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:45.309602 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:45.309499    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg" podStartSLOduration=2.9797386 podStartE2EDuration="3.309478457s" totalImagesPullingTime="329.739857ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:44:45.30589759 +0000 UTC m=+1491.073620370" watchObservedRunningTime="2026-09-10 14:44:45.309478457 +0000 UTC m=+1491.077201242"
Sep 10 14:44:51.301102 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:51.301072    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:44:55.292990 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:44:55.292956    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:45:00.580361 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:00.580315    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:06.450624 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:06.450587    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:10.243569 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:10.243524    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:15.950660 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:15.950625    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:16.220082 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:16.220003    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:16.422898 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:16.422859    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:19.715031 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:19.714997    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:19.817035 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:19.816997    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:20.917658 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:20.917625    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"]
Sep 10 14:45:20.921869 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:20.921843    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:20.923924 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:20.923904    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-dfd05fbc-serving-cert\""
Sep 10 14:45:20.932906 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:20.932884    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"]
Sep 10 14:45:20.993410 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:20.993175    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-metrics-certs\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:20.993410 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:20.993296    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0072e48e-943d-4708-9a71-33452c6d64be-openshift-service-ca\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:20.993644 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:20.993457    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2jpj4\" (UniqueName: \"kubernetes.io/projected/0072e48e-943d-4708-9a71-33452c6d64be-kube-api-access-2jpj4\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:20.993644 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:20.993492    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.026228 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.026200    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:21.095016 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.094981    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2jpj4\" (UniqueName: \"kubernetes.io/projected/0072e48e-943d-4708-9a71-33452c6d64be-kube-api-access-2jpj4\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.095201 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.095083    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.095201 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.095140    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-metrics-certs\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.095302 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.095198    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0072e48e-943d-4708-9a71-33452c6d64be-openshift-service-ca\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.095302 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:45:21.095244    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-serving-cert: secret "maas-api-e2e-mt-dfd05fbc-serving-cert" not found
Sep 10 14:45:21.095367 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:45:21.095319    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls podName:0072e48e-943d-4708-9a71-33452c6d64be nodeName:}" failed. No retries permitted until 2026-09-10 14:45:21.595299695 +0000 UTC m=+1527.363022460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls") pod "maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6" (UID: "0072e48e-943d-4708-9a71-33452c6d64be") : secret "maas-api-e2e-mt-dfd05fbc-serving-cert" not found
Sep 10 14:45:21.096031 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.096011    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0072e48e-943d-4708-9a71-33452c6d64be-openshift-service-ca\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.097888 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.097867    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-metrics-certs\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.103578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.103559    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2jpj4\" (UniqueName: \"kubernetes.io/projected/0072e48e-943d-4708-9a71-33452c6d64be-kube-api-access-2jpj4\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.124017 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.123994    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:21.401882 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.401848    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:21.516212 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.516153    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:21.598961 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.598928    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.601400 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.601381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls\") pod \"maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.612612 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.612587    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:21.654698 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.654630    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"]
Sep 10 14:45:21.659530 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.659500    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:21.662177 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.662150    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-dfd05fbc-dockercfg-8m9vw\""
Sep 10 14:45:21.662502 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.662469    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-dfd05fbc\""
Sep 10 14:45:21.670033 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.669966    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"]
Sep 10 14:45:21.700029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.699999    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b07076b-72cc-420e-92ec-84e4a8e369a9-plugins-config-volume\") pod \"payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss\" (UID: \"4b07076b-72cc-420e-92ec-84e4a8e369a9\") " pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:21.700166 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.700094    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zvtgk\" (UniqueName: \"kubernetes.io/projected/4b07076b-72cc-420e-92ec-84e4a8e369a9-kube-api-access-zvtgk\") pod \"payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss\" (UID: \"4b07076b-72cc-420e-92ec-84e4a8e369a9\") " pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:21.800855 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.800819    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zvtgk\" (UniqueName: \"kubernetes.io/projected/4b07076b-72cc-420e-92ec-84e4a8e369a9-kube-api-access-zvtgk\") pod \"payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss\" (UID: \"4b07076b-72cc-420e-92ec-84e4a8e369a9\") " pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:21.801047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.800909    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b07076b-72cc-420e-92ec-84e4a8e369a9-plugins-config-volume\") pod \"payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss\" (UID: \"4b07076b-72cc-420e-92ec-84e4a8e369a9\") " pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:21.801626 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.801601    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b07076b-72cc-420e-92ec-84e4a8e369a9-plugins-config-volume\") pod \"payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss\" (UID: \"4b07076b-72cc-420e-92ec-84e4a8e369a9\") " pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:21.808725 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.808702    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zvtgk\" (UniqueName: \"kubernetes.io/projected/4b07076b-72cc-420e-92ec-84e4a8e369a9-kube-api-access-zvtgk\") pod \"payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss\" (UID: \"4b07076b-72cc-420e-92ec-84e4a8e369a9\") " pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:21.832594 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.832567    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:21.975405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:21.975324    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:22.109057 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:22.109031    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"]
Sep 10 14:45:22.111129 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:45:22.111096    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0072e48e_943d_4708_9a71_33452c6d64be.slice/crio-716c52965484b0623a9f503fd1c03493d383b71fa74350cfbbc78c07710cb83c WatchSource:0}: Error finding container 716c52965484b0623a9f503fd1c03493d383b71fa74350cfbbc78c07710cb83c: Status 404 returned error can't find the container with id 716c52965484b0623a9f503fd1c03493d383b71fa74350cfbbc78c07710cb83c
Sep 10 14:45:22.248362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:22.248336    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"]
Sep 10 14:45:22.250164 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:45:22.250137    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b07076b_72cc_420e_92ec_84e4a8e369a9.slice/crio-c83887a4b79dfac6b34fbc07561189028deff1e45081b030c23b1da7f850be43 WatchSource:0}: Error finding container c83887a4b79dfac6b34fbc07561189028deff1e45081b030c23b1da7f850be43: Status 404 returned error can't find the container with id c83887a4b79dfac6b34fbc07561189028deff1e45081b030c23b1da7f850be43
Sep 10 14:45:22.487074 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:22.487038    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss" event={"ID":"4b07076b-72cc-420e-92ec-84e4a8e369a9","Type":"ContainerStarted","Data":"33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda"}
Sep 10 14:45:22.487279 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:22.487215    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:22.487343 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:22.487270    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss" event={"ID":"4b07076b-72cc-420e-92ec-84e4a8e369a9","Type":"ContainerStarted","Data":"c83887a4b79dfac6b34fbc07561189028deff1e45081b030c23b1da7f850be43"}
Sep 10 14:45:22.488484 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:22.488454    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6" event={"ID":"0072e48e-943d-4708-9a71-33452c6d64be","Type":"ContainerStarted","Data":"716c52965484b0623a9f503fd1c03493d383b71fa74350cfbbc78c07710cb83c"}
Sep 10 14:45:22.504767 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:22.504653    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss" podStartSLOduration=1.5046374390000001 podStartE2EDuration="1.504637439s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:45:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:45:22.502155184 +0000 UTC m=+1528.269877966" watchObservedRunningTime="2026-09-10 14:45:22.504637439 +0000 UTC m=+1528.272360260"
Sep 10 14:45:23.495088 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:23.495057    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6" event={"ID":"0072e48e-943d-4708-9a71-33452c6d64be","Type":"ContainerStarted","Data":"b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5"}
Sep 10 14:45:23.495504 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:23.495220    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:23.511880 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:23.511833    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6" podStartSLOduration=3.099024677 podStartE2EDuration="3.511818777s" totalImagesPullingTime="412.7941ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:45:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:45:23.509425941 +0000 UTC m=+1529.277148720" watchObservedRunningTime="2026-09-10 14:45:23.511818777 +0000 UTC m=+1529.279541557"
Sep 10 14:45:29.506994 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:29.506963    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:45:33.497731 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:33.497693    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:45:43.236239 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:43.236204    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:44.786919 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:44.786860    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:47.691763 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:47.691711    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:45:56.312946 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:45:56.312914    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:37.285229 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:37.285191    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:39.289467 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.289433    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"]
Sep 10 14:46:39.289944 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.289713    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6" podUID="0072e48e-943d-4708-9a71-33452c6d64be" containerName="maas-api" containerID="cri-o://b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5" gracePeriod=30
Sep 10 14:46:39.338475 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.338276    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:39.377632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.377588    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"]
Sep 10 14:46:39.377908 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.377880    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss" podUID="4b07076b-72cc-420e-92ec-84e4a8e369a9" containerName="payload-processing" containerID="cri-o://33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda" gracePeriod=30
Sep 10 14:46:39.423710 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.423676    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:39.502696 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.502652    2578 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6" podUID="0072e48e-943d-4708-9a71-33452c6d64be" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.61:8443/health\": dial tcp 10.134.0.61:8443: connect: connection refused"
Sep 10 14:46:39.525652 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.525605    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:39.625216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.625166    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:39.636353 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.636331    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:46:39.671135 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.671103    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:46:39.747063 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.747034    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0072e48e-943d-4708-9a71-33452c6d64be-openshift-service-ca\") pod \"0072e48e-943d-4708-9a71-33452c6d64be\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") "
Sep 10 14:46:39.747268 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.747085    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2jpj4\" (UniqueName: \"kubernetes.io/projected/0072e48e-943d-4708-9a71-33452c6d64be-kube-api-access-2jpj4\") pod \"0072e48e-943d-4708-9a71-33452c6d64be\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") "
Sep 10 14:46:39.747268 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.747104    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls\") pod \"0072e48e-943d-4708-9a71-33452c6d64be\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") "
Sep 10 14:46:39.747268 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.747143    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-metrics-certs\") pod \"0072e48e-943d-4708-9a71-33452c6d64be\" (UID: \"0072e48e-943d-4708-9a71-33452c6d64be\") "
Sep 10 14:46:39.747557 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.747528    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0072e48e-943d-4708-9a71-33452c6d64be-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "0072e48e-943d-4708-9a71-33452c6d64be" (UID: "0072e48e-943d-4708-9a71-33452c6d64be"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:46:39.749463 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.749431    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "0072e48e-943d-4708-9a71-33452c6d64be" (UID: "0072e48e-943d-4708-9a71-33452c6d64be"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:46:39.749573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.749458    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "0072e48e-943d-4708-9a71-33452c6d64be" (UID: "0072e48e-943d-4708-9a71-33452c6d64be"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:46:39.749573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.749483    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0072e48e-943d-4708-9a71-33452c6d64be-kube-api-access-2jpj4" (OuterVolumeSpecName: "kube-api-access-2jpj4") pod "0072e48e-943d-4708-9a71-33452c6d64be" (UID: "0072e48e-943d-4708-9a71-33452c6d64be"). InnerVolumeSpecName "kube-api-access-2jpj4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:46:39.848186 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.848151    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b07076b-72cc-420e-92ec-84e4a8e369a9-plugins-config-volume\") pod \"4b07076b-72cc-420e-92ec-84e4a8e369a9\" (UID: \"4b07076b-72cc-420e-92ec-84e4a8e369a9\") "
Sep 10 14:46:39.848379 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.848194    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zvtgk\" (UniqueName: \"kubernetes.io/projected/4b07076b-72cc-420e-92ec-84e4a8e369a9-kube-api-access-zvtgk\") pod \"4b07076b-72cc-420e-92ec-84e4a8e369a9\" (UID: \"4b07076b-72cc-420e-92ec-84e4a8e369a9\") "
Sep 10 14:46:39.848472 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.848453    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/0072e48e-943d-4708-9a71-33452c6d64be-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:46:39.848536 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.848479    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2jpj4\" (UniqueName: \"kubernetes.io/projected/0072e48e-943d-4708-9a71-33452c6d64be-kube-api-access-2jpj4\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:46:39.848536 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.848495    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:46:39.848536 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.848509    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/0072e48e-943d-4708-9a71-33452c6d64be-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:46:39.848536 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.848505    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b07076b-72cc-420e-92ec-84e4a8e369a9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b07076b-72cc-420e-92ec-84e4a8e369a9" (UID: "4b07076b-72cc-420e-92ec-84e4a8e369a9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:46:39.850408 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.850388    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b07076b-72cc-420e-92ec-84e4a8e369a9-kube-api-access-zvtgk" (OuterVolumeSpecName: "kube-api-access-zvtgk") pod "4b07076b-72cc-420e-92ec-84e4a8e369a9" (UID: "4b07076b-72cc-420e-92ec-84e4a8e369a9"). InnerVolumeSpecName "kube-api-access-zvtgk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:46:39.899324 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.899286    2578 generic.go:356] "Generic (PLEG): container finished" podID="4b07076b-72cc-420e-92ec-84e4a8e369a9" containerID="33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda" exitCode=0
Sep 10 14:46:39.899509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.899333    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"
Sep 10 14:46:39.899509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.899380    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss" event={"ID":"4b07076b-72cc-420e-92ec-84e4a8e369a9","Type":"ContainerDied","Data":"33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda"}
Sep 10 14:46:39.899509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.899438    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss" event={"ID":"4b07076b-72cc-420e-92ec-84e4a8e369a9","Type":"ContainerDied","Data":"c83887a4b79dfac6b34fbc07561189028deff1e45081b030c23b1da7f850be43"}
Sep 10 14:46:39.899509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.899469    2578 scope.go:117] "RemoveContainer" containerID="33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda"
Sep 10 14:46:39.900835 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.900814    2578 generic.go:356] "Generic (PLEG): container finished" podID="0072e48e-943d-4708-9a71-33452c6d64be" containerID="b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5" exitCode=0
Sep 10 14:46:39.900911 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.900869    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"
Sep 10 14:46:39.900961 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.900866    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6" event={"ID":"0072e48e-943d-4708-9a71-33452c6d64be","Type":"ContainerDied","Data":"b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5"}
Sep 10 14:46:39.901012 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.900996    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6" event={"ID":"0072e48e-943d-4708-9a71-33452c6d64be","Type":"ContainerDied","Data":"716c52965484b0623a9f503fd1c03493d383b71fa74350cfbbc78c07710cb83c"}
Sep 10 14:46:39.916945 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.916073    2578 scope.go:117] "RemoveContainer" containerID="33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda"
Sep 10 14:46:39.916945 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:46:39.916511    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda\": container with ID starting with 33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda not found: ID does not exist" containerID="33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda"
Sep 10 14:46:39.916945 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.916543    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda"} err="failed to get container status \"33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda\": rpc error: code = NotFound desc = could not find container \"33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda\": container with ID starting with 33c8b20ec795e7ee36a470eaa9f3d96cb1543d940b421142208f65f5f22c2dda not found: ID does not exist"
Sep 10 14:46:39.916945 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.916567    2578 scope.go:117] "RemoveContainer" containerID="b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5"
Sep 10 14:46:39.926046 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.926018    2578 scope.go:117] "RemoveContainer" containerID="b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5"
Sep 10 14:46:39.926323 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:46:39.926305    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5\": container with ID starting with b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5 not found: ID does not exist" containerID="b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5"
Sep 10 14:46:39.926392 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.926325    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5"} err="failed to get container status \"b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5\": rpc error: code = NotFound desc = could not find container \"b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5\": container with ID starting with b526bc57d0a37438aeb55a5075c9d21884513502f983c6b5eec21b72a299a9c5 not found: ID does not exist"
Sep 10 14:46:39.933108 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.933085    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"]
Sep 10 14:46:39.935058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.935039    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-dfd05fbc-75b55794dc-mwgq6"]
Sep 10 14:46:39.942585 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.942565    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"]
Sep 10 14:46:39.946571 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.946553    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-dfd05fbc-575f9858f-krpss"]
Sep 10 14:46:39.949613 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.949595    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b07076b-72cc-420e-92ec-84e4a8e369a9-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:46:39.949613 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:39.949613    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zvtgk\" (UniqueName: \"kubernetes.io/projected/4b07076b-72cc-420e-92ec-84e4a8e369a9-kube-api-access-zvtgk\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:46:40.725332 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:40.725299    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0072e48e-943d-4708-9a71-33452c6d64be" path="/var/lib/kubelet/pods/0072e48e-943d-4708-9a71-33452c6d64be/volumes"
Sep 10 14:46:40.725717 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:40.725664    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b07076b-72cc-420e-92ec-84e4a8e369a9" path="/var/lib/kubelet/pods/4b07076b-72cc-420e-92ec-84e4a8e369a9/volumes"
Sep 10 14:46:42.566566 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:42.566509    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:46:47.647718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:46:47.647633    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:17.424574 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:17.424533    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:18.534410 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:18.534349    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:24.344669 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:24.344638    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:24.615957 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:24.615869    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:30.124213 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:30.124144    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:31.967311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:31.967256    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:35.601903 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:35.601866    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:35.702499 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:35.702464    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:35.862844 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:35.862732    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:37.090513 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:37.090482    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:37.193548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:37.193515    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:37.356484 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:37.356396    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:37.906805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:37.906728    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:37.999428 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:37.999401    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:38.099508 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:38.099478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:39.296562 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:39.296525    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:45.587155 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:45.587106    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:45.686816 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:45.686779    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:46.233607 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:46.233572    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:52.373134 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:52.373095    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:52.473382 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:52.473355    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:47:54.907014 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:47:54.906975    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:02.752501 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:02.752465    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:02.965653 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:02.965615    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:23.827027 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:23.826934    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:29.439174 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:29.439137    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:29.540113 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:29.540070    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:29.736227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:29.736148    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:30.045271 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:30.045191    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:30.817325 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:30.817285    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:32.930560 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:32.930523    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:33.029514 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:33.029484    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:34.084789 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.084736    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:34.153491 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.153463    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:34.565009 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.564979    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:34.663412 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.663379    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"]
Sep 10 14:48:34.663786 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.663741    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:34.664072 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.664055    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0072e48e-943d-4708-9a71-33452c6d64be" containerName="maas-api"
Sep 10 14:48:34.664150 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.664074    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0072e48e-943d-4708-9a71-33452c6d64be" containerName="maas-api"
Sep 10 14:48:34.664150 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.664089    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b07076b-72cc-420e-92ec-84e4a8e369a9" containerName="payload-processing"
Sep 10 14:48:34.664150 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.664098    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b07076b-72cc-420e-92ec-84e4a8e369a9" containerName="payload-processing"
Sep 10 14:48:34.664295 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.664198    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0072e48e-943d-4708-9a71-33452c6d64be" containerName="maas-api"
Sep 10 14:48:34.664295 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.664212    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b07076b-72cc-420e-92ec-84e4a8e369a9" containerName="payload-processing"
Sep 10 14:48:34.672234 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.672214    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:34.674360 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.674322    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-9c318b\""
Sep 10 14:48:34.676798 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.676775    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"]
Sep 10 14:48:34.709337 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.709313    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"]
Sep 10 14:48:34.712784 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.712768    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:34.716835 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.716819    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-9c318b-dockercfg-d7d64\""
Sep 10 14:48:34.727009 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.726988    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"]
Sep 10 14:48:34.823147 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.823079    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w5n6p\" (UniqueName: \"kubernetes.io/projected/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-kube-api-access-w5n6p\") pod \"payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c\" (UID: \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:34.823147 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.823124    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c\" (UID: \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:34.823312 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.823238    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pkht\" (UniqueName: \"kubernetes.io/projected/f7f02eef-89a2-4784-889d-0690f48ed84c-kube-api-access-5pkht\") pod \"payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn\" (UID: \"f7f02eef-89a2-4784-889d-0690f48ed84c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:34.823312 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.823272    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f02eef-89a2-4784-889d-0690f48ed84c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn\" (UID: \"f7f02eef-89a2-4784-889d-0690f48ed84c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:34.923948 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.923914    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w5n6p\" (UniqueName: \"kubernetes.io/projected/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-kube-api-access-w5n6p\") pod \"payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c\" (UID: \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:34.924104 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.923958    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c\" (UID: \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:34.924104 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.924035    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pkht\" (UniqueName: \"kubernetes.io/projected/f7f02eef-89a2-4784-889d-0690f48ed84c-kube-api-access-5pkht\") pod \"payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn\" (UID: \"f7f02eef-89a2-4784-889d-0690f48ed84c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:34.924225 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.924165    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f02eef-89a2-4784-889d-0690f48ed84c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn\" (UID: \"f7f02eef-89a2-4784-889d-0690f48ed84c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:34.924696 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.924678    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-plugins-config-volume\") pod \"payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c\" (UID: \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:34.924813 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.924797    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f02eef-89a2-4784-889d-0690f48ed84c-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn\" (UID: \"f7f02eef-89a2-4784-889d-0690f48ed84c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:34.933573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.933548    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pkht\" (UniqueName: \"kubernetes.io/projected/f7f02eef-89a2-4784-889d-0690f48ed84c-kube-api-access-5pkht\") pod \"payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn\" (UID: \"f7f02eef-89a2-4784-889d-0690f48ed84c\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:34.933673 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.933550    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w5n6p\" (UniqueName: \"kubernetes.io/projected/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-kube-api-access-w5n6p\") pod \"payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c\" (UID: \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\") " pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:34.984848 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:34.984827    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:35.022504 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.022476    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:35.256191 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.256160    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"]
Sep 10 14:48:35.259291 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:48:35.259256    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf7f02eef_89a2_4784_889d_0690f48ed84c.slice/crio-94cf60a697899aaae0ce4a92f3b1c3a097fceb73a41ba7bb4c476aeeb448eb09 WatchSource:0}: Error finding container 94cf60a697899aaae0ce4a92f3b1c3a097fceb73a41ba7bb4c476aeeb448eb09: Status 404 returned error can't find the container with id 94cf60a697899aaae0ce4a92f3b1c3a097fceb73a41ba7bb4c476aeeb448eb09
Sep 10 14:48:35.308289 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.308269    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"]
Sep 10 14:48:35.309582 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:48:35.309559    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4a7374f_8bac_44f2_bc69_c7e6bc43297c.slice/crio-38140f48dc7930d3a61187c4a2acbb22c874e3dd8d17ebcdacee20d02ac482b6 WatchSource:0}: Error finding container 38140f48dc7930d3a61187c4a2acbb22c874e3dd8d17ebcdacee20d02ac482b6: Status 404 returned error can't find the container with id 38140f48dc7930d3a61187c4a2acbb22c874e3dd8d17ebcdacee20d02ac482b6
Sep 10 14:48:35.504789 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.504722    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn" event={"ID":"f7f02eef-89a2-4784-889d-0690f48ed84c","Type":"ContainerStarted","Data":"68bd738020d2cafe76f1542e91500d2d7647618f47a27984341010da212eb704"}
Sep 10 14:48:35.504975 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.504810    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn" event={"ID":"f7f02eef-89a2-4784-889d-0690f48ed84c","Type":"ContainerStarted","Data":"94cf60a697899aaae0ce4a92f3b1c3a097fceb73a41ba7bb4c476aeeb448eb09"}
Sep 10 14:48:35.504975 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.504931    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:35.506396 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.506372    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c" event={"ID":"d4a7374f-8bac-44f2-bc69-c7e6bc43297c","Type":"ContainerStarted","Data":"56508a1f6d4b71570d7fd3a4af819dd549e272a8606846919db1e07a3cdea33f"}
Sep 10 14:48:35.506503 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.506418    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:35.506503 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.506433    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c" event={"ID":"d4a7374f-8bac-44f2-bc69-c7e6bc43297c","Type":"ContainerStarted","Data":"38140f48dc7930d3a61187c4a2acbb22c874e3dd8d17ebcdacee20d02ac482b6"}
Sep 10 14:48:35.521341 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.521298    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn" podStartSLOduration=1.521284251 podStartE2EDuration="1.521284251s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:48:35.519174822 +0000 UTC m=+1721.286897601" watchObservedRunningTime="2026-09-10 14:48:35.521284251 +0000 UTC m=+1721.289007031"
Sep 10 14:48:35.536653 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:35.536618    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c" podStartSLOduration=1.536605905 podStartE2EDuration="1.536605905s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:48:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:48:35.53451713 +0000 UTC m=+1721.302239911" watchObservedRunningTime="2026-09-10 14:48:35.536605905 +0000 UTC m=+1721.304328684"
Sep 10 14:48:44.865491 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:44.865459    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:46.513899 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:46.513871    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:48:46.514307 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:46.513935    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:48:49.536515 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:49.536486    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:50.435000 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:50.434971    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:54.976061 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:54.976015    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:57.757028 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:57.756983    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:58.196057 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:58.196027    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:58.495159 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:58.495064    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:48:58.594886 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:48:58.594854    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:00.923872 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:00.923840    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:02.030556 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:02.030519    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:02.208570 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:02.208523    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:03.251716 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.251687    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:03.309285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.309252    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:03.459443 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.459415    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:03.724123 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.724085    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:03.854981 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.854953    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"]
Sep 10 14:49:03.858726 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.858705    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:03.860421 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.860396    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-84eb72-dockercfg-hbz9q\""
Sep 10 14:49:03.860421 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.860418    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-84eb72\""
Sep 10 14:49:03.870729 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.870702    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"]
Sep 10 14:49:03.955466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.955438    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q\" (UID: \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\") " pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:03.955466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:03.955469    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hs8cn\" (UniqueName: \"kubernetes.io/projected/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-kube-api-access-hs8cn\") pod \"payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q\" (UID: \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\") " pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:04.056645 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.056579    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q\" (UID: \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\") " pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:04.056645 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.056615    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hs8cn\" (UniqueName: \"kubernetes.io/projected/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-kube-api-access-hs8cn\") pod \"payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q\" (UID: \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\") " pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:04.057227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.057207    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-plugins-config-volume\") pod \"payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q\" (UID: \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\") " pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:04.063892 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.063872    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hs8cn\" (UniqueName: \"kubernetes.io/projected/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-kube-api-access-hs8cn\") pod \"payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q\" (UID: \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\") " pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:04.140483 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.140457    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:04.172505 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.172481    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:04.445216 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.445193    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"]
Sep 10 14:49:04.447989 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:49:04.447954    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc15511e2_6ddc_4fc9_bf4d_0ddf78131a8e.slice/crio-4e70972b30839f33af3058f4cc78d4d38b433ca85afa6b60669d29ef3a908839 WatchSource:0}: Error finding container 4e70972b30839f33af3058f4cc78d4d38b433ca85afa6b60669d29ef3a908839: Status 404 returned error can't find the container with id 4e70972b30839f33af3058f4cc78d4d38b433ca85afa6b60669d29ef3a908839
Sep 10 14:49:04.669978 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.669945    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q" event={"ID":"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e","Type":"ContainerStarted","Data":"a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce"}
Sep 10 14:49:04.670152 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.669997    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q" event={"ID":"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e","Type":"ContainerStarted","Data":"4e70972b30839f33af3058f4cc78d4d38b433ca85afa6b60669d29ef3a908839"}
Sep 10 14:49:04.670152 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.670062    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:04.683806 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:04.683736    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q" podStartSLOduration=1.683719817 podStartE2EDuration="1.683719817s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:49:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:49:04.682964032 +0000 UTC m=+1750.450686810" watchObservedRunningTime="2026-09-10 14:49:04.683719817 +0000 UTC m=+1750.451442600"
Sep 10 14:49:06.938330 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:06.938299    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:06.996320 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:06.996289    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:13.075312 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:13.075268    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:15.678870 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:15.678840    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:49:24.140717 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:24.140672    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:26.981676 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:26.981641    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:27.593805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:27.593778    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:27.665947 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:27.665909    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:30.044992 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:30.044958    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:31.630206 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:31.630167    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:36.085055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:36.085019    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:40.130522 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:40.130492    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:45.710621 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.710533    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:45.742631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.742605    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"]
Sep 10 14:49:45.742846 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.742820    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c" podUID="d4a7374f-8bac-44f2-bc69-c7e6bc43297c" containerName="payload-processing" containerID="cri-o://56508a1f6d4b71570d7fd3a4af819dd549e272a8606846919db1e07a3cdea33f" gracePeriod=30
Sep 10 14:49:45.750696 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.750671    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"]
Sep 10 14:49:45.751495 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.750908    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn" podUID="f7f02eef-89a2-4784-889d-0690f48ed84c" containerName="payload-pre-processing" containerID="cri-o://68bd738020d2cafe76f1542e91500d2d7647618f47a27984341010da212eb704" gracePeriod=30
Sep 10 14:49:45.793881 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.793840    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:45.862984 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.862952    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:45.886944 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.886843    2578 generic.go:356] "Generic (PLEG): container finished" podID="f7f02eef-89a2-4784-889d-0690f48ed84c" containerID="68bd738020d2cafe76f1542e91500d2d7647618f47a27984341010da212eb704" exitCode=0
Sep 10 14:49:45.886944 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.886907    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn" event={"ID":"f7f02eef-89a2-4784-889d-0690f48ed84c","Type":"ContainerDied","Data":"68bd738020d2cafe76f1542e91500d2d7647618f47a27984341010da212eb704"}
Sep 10 14:49:45.889374 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.889328    2578 generic.go:356] "Generic (PLEG): container finished" podID="d4a7374f-8bac-44f2-bc69-c7e6bc43297c" containerID="56508a1f6d4b71570d7fd3a4af819dd549e272a8606846919db1e07a3cdea33f" exitCode=0
Sep 10 14:49:45.889374 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.889361    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c" event={"ID":"d4a7374f-8bac-44f2-bc69-c7e6bc43297c","Type":"ContainerDied","Data":"56508a1f6d4b71570d7fd3a4af819dd549e272a8606846919db1e07a3cdea33f"}
Sep 10 14:49:45.929924 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.929864    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:45.993087 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:45.993018    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:46.023088 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.023067    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:49:46.026227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.026209    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:49:46.110450 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.110422    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f02eef-89a2-4784-889d-0690f48ed84c-plugins-config-volume\") pod \"f7f02eef-89a2-4784-889d-0690f48ed84c\" (UID: \"f7f02eef-89a2-4784-889d-0690f48ed84c\") "
Sep 10 14:49:46.110605 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.110488    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-plugins-config-volume\") pod \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\" (UID: \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\") "
Sep 10 14:49:46.110605 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.110539    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w5n6p\" (UniqueName: \"kubernetes.io/projected/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-kube-api-access-w5n6p\") pod \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\" (UID: \"d4a7374f-8bac-44f2-bc69-c7e6bc43297c\") "
Sep 10 14:49:46.110605 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.110598    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5pkht\" (UniqueName: \"kubernetes.io/projected/f7f02eef-89a2-4784-889d-0690f48ed84c-kube-api-access-5pkht\") pod \"f7f02eef-89a2-4784-889d-0690f48ed84c\" (UID: \"f7f02eef-89a2-4784-889d-0690f48ed84c\") "
Sep 10 14:49:46.110870 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.110848    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d4a7374f-8bac-44f2-bc69-c7e6bc43297c" (UID: "d4a7374f-8bac-44f2-bc69-c7e6bc43297c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:49:46.110870 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.110855    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f7f02eef-89a2-4784-889d-0690f48ed84c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f7f02eef-89a2-4784-889d-0690f48ed84c" (UID: "f7f02eef-89a2-4784-889d-0690f48ed84c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:49:46.112648 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.112630    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-kube-api-access-w5n6p" (OuterVolumeSpecName: "kube-api-access-w5n6p") pod "d4a7374f-8bac-44f2-bc69-c7e6bc43297c" (UID: "d4a7374f-8bac-44f2-bc69-c7e6bc43297c"). InnerVolumeSpecName "kube-api-access-w5n6p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:49:46.112704 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.112641    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f7f02eef-89a2-4784-889d-0690f48ed84c-kube-api-access-5pkht" (OuterVolumeSpecName: "kube-api-access-5pkht") pod "f7f02eef-89a2-4784-889d-0690f48ed84c" (UID: "f7f02eef-89a2-4784-889d-0690f48ed84c"). InnerVolumeSpecName "kube-api-access-5pkht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:49:46.211960 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.211934    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5pkht\" (UniqueName: \"kubernetes.io/projected/f7f02eef-89a2-4784-889d-0690f48ed84c-kube-api-access-5pkht\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:49:46.211960 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.211957    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f7f02eef-89a2-4784-889d-0690f48ed84c-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:49:46.212095 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.211965    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:49:46.212095 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.211973    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w5n6p\" (UniqueName: \"kubernetes.io/projected/d4a7374f-8bac-44f2-bc69-c7e6bc43297c-kube-api-access-w5n6p\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:49:46.286129 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.286074    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:46.895795 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.895767    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"
Sep 10 14:49:46.895795 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.895778    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn" event={"ID":"f7f02eef-89a2-4784-889d-0690f48ed84c","Type":"ContainerDied","Data":"94cf60a697899aaae0ce4a92f3b1c3a097fceb73a41ba7bb4c476aeeb448eb09"}
Sep 10 14:49:46.896209 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.895831    2578 scope.go:117] "RemoveContainer" containerID="68bd738020d2cafe76f1542e91500d2d7647618f47a27984341010da212eb704"
Sep 10 14:49:46.897271 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.897146    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c" event={"ID":"d4a7374f-8bac-44f2-bc69-c7e6bc43297c","Type":"ContainerDied","Data":"38140f48dc7930d3a61187c4a2acbb22c874e3dd8d17ebcdacee20d02ac482b6"}
Sep 10 14:49:46.897271 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.897164    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"
Sep 10 14:49:46.905054 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.905039    2578 scope.go:117] "RemoveContainer" containerID="56508a1f6d4b71570d7fd3a4af819dd549e272a8606846919db1e07a3cdea33f"
Sep 10 14:49:46.913119 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.913100    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"]
Sep 10 14:49:46.919108 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.919090    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-9c318b-575459d7f9-tlq9c"]
Sep 10 14:49:46.927795 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.927776    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"]
Sep 10 14:49:46.932234 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:46.932206    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-9c318b-6d6b457fbb-89dwn"]
Sep 10 14:49:48.724923 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:48.724886    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4a7374f-8bac-44f2-bc69-c7e6bc43297c" path="/var/lib/kubelet/pods/d4a7374f-8bac-44f2-bc69-c7e6bc43297c/volumes"
Sep 10 14:49:48.725285 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:48.725219    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f7f02eef-89a2-4784-889d-0690f48ed84c" path="/var/lib/kubelet/pods/f7f02eef-89a2-4784-889d-0690f48ed84c/volumes"
Sep 10 14:49:52.037899 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:52.037868    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:52.138125 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:52.138089    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:52.289653 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:52.289568    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:49:52.864192 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:49:52.864137    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:04.961485 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:04.961449    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:08.323937 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.323903    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:08.355192 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.355109    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"]
Sep 10 14:50:08.355453 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.355426    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q" podUID="c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e" containerName="payload-processing" containerID="cri-o://a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce" gracePeriod=30
Sep 10 14:50:08.385250 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.385223    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:08.440848 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.440817    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:08.522295 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.522267    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:08.579069 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.578991    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:08.614080 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.614050    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:50:08.685620 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.685595    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hs8cn\" (UniqueName: \"kubernetes.io/projected/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-kube-api-access-hs8cn\") pod \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\" (UID: \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\") "
Sep 10 14:50:08.685778 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.685633    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-plugins-config-volume\") pod \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\" (UID: \"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e\") "
Sep 10 14:50:08.686009 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.685979    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e" (UID: "c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:50:08.687717 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.687694    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-kube-api-access-hs8cn" (OuterVolumeSpecName: "kube-api-access-hs8cn") pod "c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e" (UID: "c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e"). InnerVolumeSpecName "kube-api-access-hs8cn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:50:08.786970 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.786946    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hs8cn\" (UniqueName: \"kubernetes.io/projected/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-kube-api-access-hs8cn\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:08.786970 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.786968    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:08.886997 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:08.886926    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:09.018954 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.018929    2578 generic.go:356] "Generic (PLEG): container finished" podID="c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e" containerID="a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce" exitCode=0
Sep 10 14:50:09.019071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.018974    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q" event={"ID":"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e","Type":"ContainerDied","Data":"a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce"}
Sep 10 14:50:09.019071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.018979    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"
Sep 10 14:50:09.019071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.019014    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q" event={"ID":"c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e","Type":"ContainerDied","Data":"4e70972b30839f33af3058f4cc78d4d38b433ca85afa6b60669d29ef3a908839"}
Sep 10 14:50:09.019071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.019036    2578 scope.go:117] "RemoveContainer" containerID="a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce"
Sep 10 14:50:09.027411 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.027398    2578 scope.go:117] "RemoveContainer" containerID="a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce"
Sep 10 14:50:09.027680 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:50:09.027663    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce\": container with ID starting with a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce not found: ID does not exist" containerID="a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce"
Sep 10 14:50:09.027731 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.027688    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce"} err="failed to get container status \"a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce\": rpc error: code = NotFound desc = could not find container \"a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce\": container with ID starting with a149870138a01138db931d9850eeb1177ec4c923f79bc1617f2caf4b449977ce not found: ID does not exist"
Sep 10 14:50:09.034604 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.034585    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"]
Sep 10 14:50:09.038860 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:09.038842    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-84eb72-cf68c6c49-r2w4q"]
Sep 10 14:50:10.724722 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:10.724690    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e" path="/var/lib/kubelet/pods/c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e/volumes"
Sep 10 14:50:14.468803 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:14.468765    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:14.569383 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:14.569349    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:14.728354 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:14.728277    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:15.501823 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:15.501788    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:20.918871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:20.918838    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:21.834623 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:21.834592    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:22.275447 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:22.275359    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:22.558999 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:22.558895    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:22.744086 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:22.744053    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:25.566128 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:25.566084    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:25.655942 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:25.655900    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:25.706727 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:25.706697    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"]
Sep 10 14:50:25.707221 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:25.707190    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8" podUID="8718a975-1b60-40c8-96bf-0890ba8e50a8" containerName="payload-pre-processing" containerID="cri-o://076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a" gracePeriod=30
Sep 10 14:50:25.739943 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:25.739919    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:25.860828 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:25.860793    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:25.954362 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:25.954342    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:50:25.991445 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:25.991416    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:26.107614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.107534    2578 generic.go:356] "Generic (PLEG): container finished" podID="8718a975-1b60-40c8-96bf-0890ba8e50a8" containerID="076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a" exitCode=0
Sep 10 14:50:26.107614 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.107578    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"
Sep 10 14:50:26.107805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.107626    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8" event={"ID":"8718a975-1b60-40c8-96bf-0890ba8e50a8","Type":"ContainerDied","Data":"076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a"}
Sep 10 14:50:26.107805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.107676    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8" event={"ID":"8718a975-1b60-40c8-96bf-0890ba8e50a8","Type":"ContainerDied","Data":"86712ad9ac4355c1b364765a929d44421382576616298e56624be6a18ee47462"}
Sep 10 14:50:26.107805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.107699    2578 scope.go:117] "RemoveContainer" containerID="076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a"
Sep 10 14:50:26.116242 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.116226    2578 scope.go:117] "RemoveContainer" containerID="076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a"
Sep 10 14:50:26.116514 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:50:26.116494    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a\": container with ID starting with 076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a not found: ID does not exist" containerID="076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a"
Sep 10 14:50:26.116558 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.116524    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a"} err="failed to get container status \"076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a\": rpc error: code = NotFound desc = could not find container \"076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a\": container with ID starting with 076424fdc2bda48e62f9221cf286d05b7733f208ed0bb6a6c1dad9f7f556923a not found: ID does not exist"
Sep 10 14:50:26.122825 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.122806    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fnqln\" (UniqueName: \"kubernetes.io/projected/8718a975-1b60-40c8-96bf-0890ba8e50a8-kube-api-access-fnqln\") pod \"8718a975-1b60-40c8-96bf-0890ba8e50a8\" (UID: \"8718a975-1b60-40c8-96bf-0890ba8e50a8\") "
Sep 10 14:50:26.122940 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.122922    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8718a975-1b60-40c8-96bf-0890ba8e50a8-plugins-config-volume\") pod \"8718a975-1b60-40c8-96bf-0890ba8e50a8\" (UID: \"8718a975-1b60-40c8-96bf-0890ba8e50a8\") "
Sep 10 14:50:26.123237 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.123217    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8718a975-1b60-40c8-96bf-0890ba8e50a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8718a975-1b60-40c8-96bf-0890ba8e50a8" (UID: "8718a975-1b60-40c8-96bf-0890ba8e50a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:50:26.124872 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.124852    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8718a975-1b60-40c8-96bf-0890ba8e50a8-kube-api-access-fnqln" (OuterVolumeSpecName: "kube-api-access-fnqln") pod "8718a975-1b60-40c8-96bf-0890ba8e50a8" (UID: "8718a975-1b60-40c8-96bf-0890ba8e50a8"). InnerVolumeSpecName "kube-api-access-fnqln". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:50:26.224196 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.224172    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8718a975-1b60-40c8-96bf-0890ba8e50a8-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:26.224196 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.224194    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fnqln\" (UniqueName: \"kubernetes.io/projected/8718a975-1b60-40c8-96bf-0890ba8e50a8-kube-api-access-fnqln\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:26.306013 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.305993    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:26.428140 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.428113    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"]
Sep 10 14:50:26.434592 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.434570    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-bc9af3-84664d596b-j8sr8"]
Sep 10 14:50:26.724247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.724175    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8718a975-1b60-40c8-96bf-0890ba8e50a8" path="/var/lib/kubelet/pods/8718a975-1b60-40c8-96bf-0890ba8e50a8/volumes"
Sep 10 14:50:26.796954 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.796918    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:26.895378 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:26.895347    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:27.147182 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.147157    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:27.286137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.286102    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:27.394330 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394300    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"]
Sep 10 14:50:27.394482 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394404    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:27.394774 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394738    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e" containerName="payload-processing"
Sep 10 14:50:27.394774 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394771    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e" containerName="payload-processing"
Sep 10 14:50:27.394853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394810    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4a7374f-8bac-44f2-bc69-c7e6bc43297c" containerName="payload-processing"
Sep 10 14:50:27.394853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394819    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4a7374f-8bac-44f2-bc69-c7e6bc43297c" containerName="payload-processing"
Sep 10 14:50:27.394853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394829    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8718a975-1b60-40c8-96bf-0890ba8e50a8" containerName="payload-pre-processing"
Sep 10 14:50:27.394853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394835    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="8718a975-1b60-40c8-96bf-0890ba8e50a8" containerName="payload-pre-processing"
Sep 10 14:50:27.394853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394844    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f7f02eef-89a2-4784-889d-0690f48ed84c" containerName="payload-pre-processing"
Sep 10 14:50:27.394853 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394849    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f7f02eef-89a2-4784-889d-0690f48ed84c" containerName="payload-pre-processing"
Sep 10 14:50:27.395027 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394927    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f7f02eef-89a2-4784-889d-0690f48ed84c" containerName="payload-pre-processing"
Sep 10 14:50:27.395027 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394935    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="8718a975-1b60-40c8-96bf-0890ba8e50a8" containerName="payload-pre-processing"
Sep 10 14:50:27.395027 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394942    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c15511e2-6ddc-4fc9-bf4d-0ddf78131a8e" containerName="payload-processing"
Sep 10 14:50:27.395027 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.394948    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4a7374f-8bac-44f2-bc69-c7e6bc43297c" containerName="payload-processing"
Sep 10 14:50:27.399461 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.399418    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:27.401327 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.401302    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-cc50fd-dockercfg-rgnv4\""
Sep 10 14:50:27.401442 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.401330    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-cc50fd\""
Sep 10 14:50:27.404786 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.404765    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"]
Sep 10 14:50:27.538318 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.538290    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h5s5k\" (UniqueName: \"kubernetes.io/projected/5a514fd2-e9e3-4148-8df9-ba560707c9ef-kube-api-access-h5s5k\") pod \"payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt\" (UID: \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:27.538472 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.538329    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a514fd2-e9e3-4148-8df9-ba560707c9ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt\" (UID: \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:27.638940 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.638910    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h5s5k\" (UniqueName: \"kubernetes.io/projected/5a514fd2-e9e3-4148-8df9-ba560707c9ef-kube-api-access-h5s5k\") pod \"payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt\" (UID: \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:27.639064 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.638952    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a514fd2-e9e3-4148-8df9-ba560707c9ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt\" (UID: \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:27.639557 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.639538    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a514fd2-e9e3-4148-8df9-ba560707c9ef-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt\" (UID: \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:27.646636 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.646608    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h5s5k\" (UniqueName: \"kubernetes.io/projected/5a514fd2-e9e3-4148-8df9-ba560707c9ef-kube-api-access-h5s5k\") pod \"payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt\" (UID: \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:27.710799 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.710719    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:27.981385 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:27.981360    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"]
Sep 10 14:50:27.983778 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:50:27.983732    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a514fd2_e9e3_4148_8df9_ba560707c9ef.slice/crio-200cef9bf16048170318dbc841503bd8ecd220378a2252cf720d7f7d723fe4ef WatchSource:0}: Error finding container 200cef9bf16048170318dbc841503bd8ecd220378a2252cf720d7f7d723fe4ef: Status 404 returned error can't find the container with id 200cef9bf16048170318dbc841503bd8ecd220378a2252cf720d7f7d723fe4ef
Sep 10 14:50:28.121209 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:28.121171    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt" event={"ID":"5a514fd2-e9e3-4148-8df9-ba560707c9ef","Type":"ContainerStarted","Data":"f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e"}
Sep 10 14:50:28.121378 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:28.121233    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt" event={"ID":"5a514fd2-e9e3-4148-8df9-ba560707c9ef","Type":"ContainerStarted","Data":"200cef9bf16048170318dbc841503bd8ecd220378a2252cf720d7f7d723fe4ef"}
Sep 10 14:50:28.121378 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:28.121313    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:28.136089 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:28.136042    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt" podStartSLOduration=1.136029508 podStartE2EDuration="1.136029508s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:50:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:50:28.134921717 +0000 UTC m=+1833.902644502" watchObservedRunningTime="2026-09-10 14:50:28.136029508 +0000 UTC m=+1833.903752306"
Sep 10 14:50:31.985583 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:31.985556    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:32.048145 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:32.048117    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:32.245807 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:32.245715    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:32.853644 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:32.853602    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:39.129276 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:39.129243    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:50:43.158558 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.158525    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"]
Sep 10 14:50:43.159034 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.158772    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg" podUID="46b9fc5e-2632-4e29-a87a-8c30a505fe76" containerName="maas-api" containerID="cri-o://0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e" gracePeriod=30
Sep 10 14:50:43.185905 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.185874    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:43.220145 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.220117    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"]
Sep 10 14:50:43.220385 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.220359    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99" podUID="851ae358-b268-480f-9673-92f449cd705c" containerName="payload-processing" containerID="cri-o://5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c" gracePeriod=30
Sep 10 14:50:43.257321 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.257290    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:43.329836 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.329809    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:43.408461 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.408430    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:43.470357 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.470327    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:43.481549 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.481526    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:50:43.506695 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.506670    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:50:43.550567 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.550544    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls\") pod \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") "
Sep 10 14:50:43.550696 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.550603    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rf8fx\" (UniqueName: \"kubernetes.io/projected/851ae358-b268-480f-9673-92f449cd705c-kube-api-access-rf8fx\") pod \"851ae358-b268-480f-9673-92f449cd705c\" (UID: \"851ae358-b268-480f-9673-92f449cd705c\") "
Sep 10 14:50:43.550696 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.550643    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46b9fc5e-2632-4e29-a87a-8c30a505fe76-openshift-service-ca\") pod \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") "
Sep 10 14:50:43.550838 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.550701    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-metrics-certs\") pod \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") "
Sep 10 14:50:43.550838 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.550734    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851ae358-b268-480f-9673-92f449cd705c-plugins-config-volume\") pod \"851ae358-b268-480f-9673-92f449cd705c\" (UID: \"851ae358-b268-480f-9673-92f449cd705c\") "
Sep 10 14:50:43.550838 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.550797    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7fdjd\" (UniqueName: \"kubernetes.io/projected/46b9fc5e-2632-4e29-a87a-8c30a505fe76-kube-api-access-7fdjd\") pod \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\" (UID: \"46b9fc5e-2632-4e29-a87a-8c30a505fe76\") "
Sep 10 14:50:43.551186 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.551074    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/46b9fc5e-2632-4e29-a87a-8c30a505fe76-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "46b9fc5e-2632-4e29-a87a-8c30a505fe76" (UID: "46b9fc5e-2632-4e29-a87a-8c30a505fe76"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:50:43.551558 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.551421    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/851ae358-b268-480f-9673-92f449cd705c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "851ae358-b268-480f-9673-92f449cd705c" (UID: "851ae358-b268-480f-9673-92f449cd705c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:50:43.552960 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.552936    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "46b9fc5e-2632-4e29-a87a-8c30a505fe76" (UID: "46b9fc5e-2632-4e29-a87a-8c30a505fe76"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:50:43.553202 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.553178    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/46b9fc5e-2632-4e29-a87a-8c30a505fe76-kube-api-access-7fdjd" (OuterVolumeSpecName: "kube-api-access-7fdjd") pod "46b9fc5e-2632-4e29-a87a-8c30a505fe76" (UID: "46b9fc5e-2632-4e29-a87a-8c30a505fe76"). InnerVolumeSpecName "kube-api-access-7fdjd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:50:43.553364 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.553350    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/851ae358-b268-480f-9673-92f449cd705c-kube-api-access-rf8fx" (OuterVolumeSpecName: "kube-api-access-rf8fx") pod "851ae358-b268-480f-9673-92f449cd705c" (UID: "851ae358-b268-480f-9673-92f449cd705c"). InnerVolumeSpecName "kube-api-access-rf8fx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:50:43.553516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.553494    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "46b9fc5e-2632-4e29-a87a-8c30a505fe76" (UID: "46b9fc5e-2632-4e29-a87a-8c30a505fe76"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:50:43.652137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.652118    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:43.652137 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.652136    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rf8fx\" (UniqueName: \"kubernetes.io/projected/851ae358-b268-480f-9673-92f449cd705c-kube-api-access-rf8fx\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:43.652259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.652146    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/46b9fc5e-2632-4e29-a87a-8c30a505fe76-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:43.652259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.652155    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/46b9fc5e-2632-4e29-a87a-8c30a505fe76-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:43.652259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.652164    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/851ae358-b268-480f-9673-92f449cd705c-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:43.652259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.652172    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7fdjd\" (UniqueName: \"kubernetes.io/projected/46b9fc5e-2632-4e29-a87a-8c30a505fe76-kube-api-access-7fdjd\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:50:43.779631 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:43.779571    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:44.210207 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.210172    2578 generic.go:356] "Generic (PLEG): container finished" podID="851ae358-b268-480f-9673-92f449cd705c" containerID="5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c" exitCode=0
Sep 10 14:50:44.210670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.210218    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"
Sep 10 14:50:44.210670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.210259    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99" event={"ID":"851ae358-b268-480f-9673-92f449cd705c","Type":"ContainerDied","Data":"5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c"}
Sep 10 14:50:44.210670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.210324    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99" event={"ID":"851ae358-b268-480f-9673-92f449cd705c","Type":"ContainerDied","Data":"16941dd47a5dbce5bc30349ce69852e6397ee2a82d5d65f2b3e9926d30e66c8c"}
Sep 10 14:50:44.210670 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.210359    2578 scope.go:117] "RemoveContainer" containerID="5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c"
Sep 10 14:50:44.211973 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.211923    2578 generic.go:356] "Generic (PLEG): container finished" podID="46b9fc5e-2632-4e29-a87a-8c30a505fe76" containerID="0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e" exitCode=0
Sep 10 14:50:44.212094 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.211975    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"
Sep 10 14:50:44.212094 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.211983    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg" event={"ID":"46b9fc5e-2632-4e29-a87a-8c30a505fe76","Type":"ContainerDied","Data":"0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e"}
Sep 10 14:50:44.212094 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.212021    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg" event={"ID":"46b9fc5e-2632-4e29-a87a-8c30a505fe76","Type":"ContainerDied","Data":"ee26b612f22ed929fa0e58704326eb6212835265e03f68451387c8314ce21f82"}
Sep 10 14:50:44.221833 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.221810    2578 scope.go:117] "RemoveContainer" containerID="5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c"
Sep 10 14:50:44.222169 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:50:44.222150    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c\": container with ID starting with 5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c not found: ID does not exist" containerID="5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c"
Sep 10 14:50:44.222217 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.222179    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c"} err="failed to get container status \"5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c\": rpc error: code = NotFound desc = could not find container \"5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c\": container with ID starting with 5c93e7ae979d28d0829c81902df3fd90dcd9cb9735756311fe8701cc92df388c not found: ID does not exist"
Sep 10 14:50:44.222217 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.222196    2578 scope.go:117] "RemoveContainer" containerID="0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e"
Sep 10 14:50:44.233443 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.233424    2578 scope.go:117] "RemoveContainer" containerID="0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e"
Sep 10 14:50:44.233716 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:50:44.233698    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e\": container with ID starting with 0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e not found: ID does not exist" containerID="0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e"
Sep 10 14:50:44.233795 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.233722    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e"} err="failed to get container status \"0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e\": rpc error: code = NotFound desc = could not find container \"0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e\": container with ID starting with 0790eb1b0f0b51ce472c815728527ba296ca17aedabea9886d2d1d8b0f8b393e not found: ID does not exist"
Sep 10 14:50:44.245051 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.245028    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"]
Sep 10 14:50:44.250572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.250553    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w2-86a2c6-9df568cf9-6bj99"]
Sep 10 14:50:44.259845 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.259825    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"]
Sep 10 14:50:44.261426 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.261408    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-86a2c6-56c6995dd6-szfcg"]
Sep 10 14:50:44.724476 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.724442    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="46b9fc5e-2632-4e29-a87a-8c30a505fe76" path="/var/lib/kubelet/pods/46b9fc5e-2632-4e29-a87a-8c30a505fe76/volumes"
Sep 10 14:50:44.724800 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:44.724786    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="851ae358-b268-480f-9673-92f449cd705c" path="/var/lib/kubelet/pods/851ae358-b268-480f-9673-92f449cd705c/volumes"
Sep 10 14:50:47.678358 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:47.678320    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:49.195319 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:49.195268    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:49.660846 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:49.660803    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:49.725271 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:49.725245    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:49.954480 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:49.954401    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:50.509291 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:50.509257    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:51.176708 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:51.176660    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:51.488272 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:51.488175    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:51.553010 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:51.552982    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:53.041776 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:53.041722    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:54.693004 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:54.692970    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:54.776731 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:54.776692    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:55.797411 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.797383    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"]
Sep 10 14:50:55.797811 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.797795    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="46b9fc5e-2632-4e29-a87a-8c30a505fe76" containerName="maas-api"
Sep 10 14:50:55.797811 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.797806    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="46b9fc5e-2632-4e29-a87a-8c30a505fe76" containerName="maas-api"
Sep 10 14:50:55.797907 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.797821    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="851ae358-b268-480f-9673-92f449cd705c" containerName="payload-processing"
Sep 10 14:50:55.797907 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.797827    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="851ae358-b268-480f-9673-92f449cd705c" containerName="payload-processing"
Sep 10 14:50:55.797907 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.797891    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="46b9fc5e-2632-4e29-a87a-8c30a505fe76" containerName="maas-api"
Sep 10 14:50:55.797907 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.797902    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="851ae358-b268-480f-9673-92f449cd705c" containerName="payload-processing"
Sep 10 14:50:55.802454 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.802434    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.804390 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.804368    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-eaf326-serving-cert\""
Sep 10 14:50:55.809887 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.809865    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"]
Sep 10 14:50:55.842512 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.842485    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:55.848553 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.848534    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.848660 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.848620    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621970f6-4aaf-4d46-ae90-a713bb7e6daf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.848660 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.848640    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-metrics-certs\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.848660 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.848655    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtq9p\" (UniqueName: \"kubernetes.io/projected/621970f6-4aaf-4d46-ae90-a713bb7e6daf-kube-api-access-xtq9p\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.943585 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.943563    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:55.949681 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.949661    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.949789 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.949765    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621970f6-4aaf-4d46-ae90-a713bb7e6daf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.949840 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.949791    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-metrics-certs\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.949840 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:50:55.949804    2578 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-serving-cert: secret "maas-api-e2e-ipp-b-eaf326-serving-cert" not found
Sep 10 14:50:55.949904 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.949809    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtq9p\" (UniqueName: \"kubernetes.io/projected/621970f6-4aaf-4d46-ae90-a713bb7e6daf-kube-api-access-xtq9p\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.949904 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:50:55.949851    2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls podName:621970f6-4aaf-4d46-ae90-a713bb7e6daf nodeName:}" failed. No retries permitted until 2026-09-10 14:50:56.449837055 +0000 UTC m=+1862.217559837 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls") pod "maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd" (UID: "621970f6-4aaf-4d46-ae90-a713bb7e6daf") : secret "maas-api-e2e-ipp-b-eaf326-serving-cert" not found
Sep 10 14:50:55.950396 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.950374    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621970f6-4aaf-4d46-ae90-a713bb7e6daf-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.952328 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.952305    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-metrics-certs\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:55.957912 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:55.957894    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtq9p\" (UniqueName: \"kubernetes.io/projected/621970f6-4aaf-4d46-ae90-a713bb7e6daf-kube-api-access-xtq9p\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:56.087419 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:56.087386    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:56.394376 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:56.394295    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:56.454484 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:56.454452    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:56.456779 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:56.456759    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls\") pod \"maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:56.651233 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:56.651154    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:50:56.716158 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:56.716131    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:56.994076 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:56.993985    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"]
Sep 10 14:50:56.997060 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:50:56.997028    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod621970f6_4aaf_4d46_ae90_a713bb7e6daf.slice/crio-cb5b6bbb1e3729b46c200cede88555f77d50f5b10b66935a829f43b8f08e5291 WatchSource:0}: Error finding container cb5b6bbb1e3729b46c200cede88555f77d50f5b10b66935a829f43b8f08e5291: Status 404 returned error can't find the container with id cb5b6bbb1e3729b46c200cede88555f77d50f5b10b66935a829f43b8f08e5291
Sep 10 14:50:56.998652 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:56.998631    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:50:57.287407 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:57.287322    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd" event={"ID":"621970f6-4aaf-4d46-ae90-a713bb7e6daf","Type":"ContainerStarted","Data":"cb5b6bbb1e3729b46c200cede88555f77d50f5b10b66935a829f43b8f08e5291"}
Sep 10 14:50:58.293946 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:58.293912    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd" event={"ID":"621970f6-4aaf-4d46-ae90-a713bb7e6daf","Type":"ContainerStarted","Data":"d28af495794df55e96c568cf90003c7ec988acfbdc168d0ccc6ec20aa724c567"}
Sep 10 14:50:58.294311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:58.293972    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:50:58.325037 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:50:58.324990    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd" podStartSLOduration=2.769017114 podStartE2EDuration="3.324973242s" totalImagesPullingTime="555.956128ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:50:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:50:58.3228426 +0000 UTC m=+1864.090565380" watchObservedRunningTime="2026-09-10 14:50:58.324973242 +0000 UTC m=+1864.092696037"
Sep 10 14:51:04.304000 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:04.303973    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:51:19.630953 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:19.630856    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:24.502166 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:24.502137    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vjxxl_c4ce8ee1-18ff-495f-a1d6-20320ed2118d/payload-processing/0.log"
Sep 10 14:51:26.554007 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:26.553964    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:26.603874 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:26.603847    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:28.943602 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:28.943569    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:37.993600 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:37.993560    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:55.163883 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:55.163855    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vjxxl_c4ce8ee1-18ff-495f-a1d6-20320ed2118d/payload-processing/0.log"
Sep 10 14:51:55.555718 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:55.555645    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:55.842268 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:55.842234    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:55.939625 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:55.939596    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:59.076915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:59.076871    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:51:59.178641 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:51:59.178603    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:00.334324 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.334290    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"]
Sep 10 14:52:00.338643 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.338621    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.340278 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.340257    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-3876ff-serving-cert\""
Sep 10 14:52:00.347247 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.347227    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"]
Sep 10 14:52:00.365367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.365345    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:00.442802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.442778    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.442915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.442813    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.442915 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.442834    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgk2t\" (UniqueName: \"kubernetes.io/projected/df26b3f9-5b87-4125-aa50-7870a0fd2caa-kube-api-access-cgk2t\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.443032 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.442936    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df26b3f9-5b87-4125-aa50-7870a0fd2caa-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.467987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.467950    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:00.543702 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.543679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.543823 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.543713    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgk2t\" (UniqueName: \"kubernetes.io/projected/df26b3f9-5b87-4125-aa50-7870a0fd2caa-kube-api-access-cgk2t\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.543823 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.543789    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df26b3f9-5b87-4125-aa50-7870a0fd2caa-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.543946 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.543895    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.544390 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.544368    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df26b3f9-5b87-4125-aa50-7870a0fd2caa-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.546375 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.546353    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.546465 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.546381    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.550842 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.550823    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgk2t\" (UniqueName: \"kubernetes.io/projected/df26b3f9-5b87-4125-aa50-7870a0fd2caa-kube-api-access-cgk2t\") pod \"maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.565761 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.565718    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:00.650173 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.650123    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:00.848880 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.848849    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:00.924923 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.924893    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"]
Sep 10 14:52:00.926980 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.926953    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:00.928036 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:52:00.928012    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf26b3f9_5b87_4125_aa50_7870a0fd2caa.slice/crio-7d5a83101681bcdc822a2995be84050d4927e42387e2dc886c9ff90e083d996a WatchSource:0}: Error finding container 7d5a83101681bcdc822a2995be84050d4927e42387e2dc886c9ff90e083d996a: Status 404 returned error can't find the container with id 7d5a83101681bcdc822a2995be84050d4927e42387e2dc886c9ff90e083d996a
Sep 10 14:52:00.959725 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.959699    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"]
Sep 10 14:52:00.964222 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.964204    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:00.967767 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.966955    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-3876ff-dockercfg-6f2qn\""
Sep 10 14:52:00.967767 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.967095    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-3876ff\""
Sep 10 14:52:00.983459 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.983438    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"]
Sep 10 14:52:00.997541 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:00.997521    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"]
Sep 10 14:52:01.001186 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.001166    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:01.011523 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.011501    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"]
Sep 10 14:52:01.048805 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.048771    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32927f99-9dad-44e8-8199-f65afb0fcc39-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9\" (UID: \"32927f99-9dad-44e8-8199-f65afb0fcc39\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:01.048911 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.048891    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wsvb\" (UniqueName: \"kubernetes.io/projected/346aed8f-f0dc-44ab-93f4-1dec902cad72-kube-api-access-7wsvb\") pod \"payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48\" (UID: \"346aed8f-f0dc-44ab-93f4-1dec902cad72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:01.048958 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.048937    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-plxpb\" (UniqueName: \"kubernetes.io/projected/32927f99-9dad-44e8-8199-f65afb0fcc39-kube-api-access-plxpb\") pod \"payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9\" (UID: \"32927f99-9dad-44e8-8199-f65afb0fcc39\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:01.048958 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.048955    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346aed8f-f0dc-44ab-93f4-1dec902cad72-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48\" (UID: \"346aed8f-f0dc-44ab-93f4-1dec902cad72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:01.149714 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.149679    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-plxpb\" (UniqueName: \"kubernetes.io/projected/32927f99-9dad-44e8-8199-f65afb0fcc39-kube-api-access-plxpb\") pod \"payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9\" (UID: \"32927f99-9dad-44e8-8199-f65afb0fcc39\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:01.149887 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.149718    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346aed8f-f0dc-44ab-93f4-1dec902cad72-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48\" (UID: \"346aed8f-f0dc-44ab-93f4-1dec902cad72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:01.149887 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.149821    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32927f99-9dad-44e8-8199-f65afb0fcc39-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9\" (UID: \"32927f99-9dad-44e8-8199-f65afb0fcc39\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:01.150004 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.149925    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wsvb\" (UniqueName: \"kubernetes.io/projected/346aed8f-f0dc-44ab-93f4-1dec902cad72-kube-api-access-7wsvb\") pod \"payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48\" (UID: \"346aed8f-f0dc-44ab-93f4-1dec902cad72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:01.150445 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.150422    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346aed8f-f0dc-44ab-93f4-1dec902cad72-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48\" (UID: \"346aed8f-f0dc-44ab-93f4-1dec902cad72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:01.150522 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.150450    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32927f99-9dad-44e8-8199-f65afb0fcc39-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9\" (UID: \"32927f99-9dad-44e8-8199-f65afb0fcc39\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:01.159707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.159683    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-plxpb\" (UniqueName: \"kubernetes.io/projected/32927f99-9dad-44e8-8199-f65afb0fcc39-kube-api-access-plxpb\") pod \"payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9\" (UID: \"32927f99-9dad-44e8-8199-f65afb0fcc39\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:01.159800 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.159716    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wsvb\" (UniqueName: \"kubernetes.io/projected/346aed8f-f0dc-44ab-93f4-1dec902cad72-kube-api-access-7wsvb\") pod \"payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48\" (UID: \"346aed8f-f0dc-44ab-93f4-1dec902cad72\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:01.276429 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.276362    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:01.311568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.311542    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:01.549825 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.549727    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"]
Sep 10 14:52:01.553006 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:52:01.552979    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32927f99_9dad_44e8_8199_f65afb0fcc39.slice/crio-5d4ff360e816a67f0cf90077d6d5c829492634f57873d73f0f09dc331cc3e8ac WatchSource:0}: Error finding container 5d4ff360e816a67f0cf90077d6d5c829492634f57873d73f0f09dc331cc3e8ac: Status 404 returned error can't find the container with id 5d4ff360e816a67f0cf90077d6d5c829492634f57873d73f0f09dc331cc3e8ac
Sep 10 14:52:01.596625 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.596602    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"]
Sep 10 14:52:01.599038 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:52:01.599008    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod346aed8f_f0dc_44ab_93f4_1dec902cad72.slice/crio-a8bdcab491ecd5cdfc9055c85206239d89a20fb9bc2f04e4ab2f690652e6c8b7 WatchSource:0}: Error finding container a8bdcab491ecd5cdfc9055c85206239d89a20fb9bc2f04e4ab2f690652e6c8b7: Status 404 returned error can't find the container with id a8bdcab491ecd5cdfc9055c85206239d89a20fb9bc2f04e4ab2f690652e6c8b7
Sep 10 14:52:01.632648 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.632558    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9" event={"ID":"32927f99-9dad-44e8-8199-f65afb0fcc39","Type":"ContainerStarted","Data":"5d4ff360e816a67f0cf90077d6d5c829492634f57873d73f0f09dc331cc3e8ac"}
Sep 10 14:52:01.634945 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.634896    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh" event={"ID":"df26b3f9-5b87-4125-aa50-7870a0fd2caa","Type":"ContainerStarted","Data":"353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb"}
Sep 10 14:52:01.635071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.634963    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh" event={"ID":"df26b3f9-5b87-4125-aa50-7870a0fd2caa","Type":"ContainerStarted","Data":"7d5a83101681bcdc822a2995be84050d4927e42387e2dc886c9ff90e083d996a"}
Sep 10 14:52:01.635322 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.635298    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:01.636711 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.636689    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48" event={"ID":"346aed8f-f0dc-44ab-93f4-1dec902cad72","Type":"ContainerStarted","Data":"a8bdcab491ecd5cdfc9055c85206239d89a20fb9bc2f04e4ab2f690652e6c8b7"}
Sep 10 14:52:01.651015 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:01.650966    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh" podStartSLOduration=1.177827241 podStartE2EDuration="1.650951415s" totalImagesPullingTime="473.124174ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:52:01.649971925 +0000 UTC m=+1927.417694716" watchObservedRunningTime="2026-09-10 14:52:01.650951415 +0000 UTC m=+1927.418674195"
Sep 10 14:52:02.643586 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:02.643543    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9" event={"ID":"32927f99-9dad-44e8-8199-f65afb0fcc39","Type":"ContainerStarted","Data":"ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7"}
Sep 10 14:52:02.644018 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:02.643612    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:02.645168 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:02.645139    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48" event={"ID":"346aed8f-f0dc-44ab-93f4-1dec902cad72","Type":"ContainerStarted","Data":"3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d"}
Sep 10 14:52:02.659259 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:02.659205    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9" podStartSLOduration=2.659186276 podStartE2EDuration="2.659186276s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:52:02.656665261 +0000 UTC m=+1928.424388040" watchObservedRunningTime="2026-09-10 14:52:02.659186276 +0000 UTC m=+1928.426909061"
Sep 10 14:52:02.671029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:02.670985    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48" podStartSLOduration=2.670968485 podStartE2EDuration="2.670968485s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:52:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:52:02.66957707 +0000 UTC m=+1928.437299849" watchObservedRunningTime="2026-09-10 14:52:02.670968485 +0000 UTC m=+1928.438691266"
Sep 10 14:52:03.649958 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:03.649932    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:07.649987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:07.649953    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:13.652249 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:13.652217    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:14.657560 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:14.657532    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:24.091715 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:24.091680    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:30.820102 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:30.820070    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"]
Sep 10 14:52:30.820650 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:30.820286    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh" podUID="df26b3f9-5b87-4125-aa50-7870a0fd2caa" containerName="maas-api" containerID="cri-o://353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb" gracePeriod=30
Sep 10 14:52:30.851634 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:30.851570    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:30.873023 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:30.872993    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"]
Sep 10 14:52:30.873297 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:30.873270    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48" podUID="346aed8f-f0dc-44ab-93f4-1dec902cad72" containerName="payload-processing" containerID="cri-o://3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d" gracePeriod=30
Sep 10 14:52:30.878809 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:30.878784    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"]
Sep 10 14:52:30.878986 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:30.878953    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9" podUID="32927f99-9dad-44e8-8199-f65afb0fcc39" containerName="payload-pre-processing" containerID="cri-o://ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7" gracePeriod=30
Sep 10 14:52:30.916788 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:30.916737    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:31.019946 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.019899    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:31.121058 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.121028    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:31.138445 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.138423    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:31.176185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.176165    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:31.181460 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.181438    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:31.290939 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.290912    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-maas-api-tls\") pod \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") "
Sep 10 14:52:31.290939 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.290942    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-plxpb\" (UniqueName: \"kubernetes.io/projected/32927f99-9dad-44e8-8199-f65afb0fcc39-kube-api-access-plxpb\") pod \"32927f99-9dad-44e8-8199-f65afb0fcc39\" (UID: \"32927f99-9dad-44e8-8199-f65afb0fcc39\") "
Sep 10 14:52:31.291185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.290960    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df26b3f9-5b87-4125-aa50-7870a0fd2caa-openshift-service-ca\") pod \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") "
Sep 10 14:52:31.291185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.291001    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wsvb\" (UniqueName: \"kubernetes.io/projected/346aed8f-f0dc-44ab-93f4-1dec902cad72-kube-api-access-7wsvb\") pod \"346aed8f-f0dc-44ab-93f4-1dec902cad72\" (UID: \"346aed8f-f0dc-44ab-93f4-1dec902cad72\") "
Sep 10 14:52:31.291185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.291109    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgk2t\" (UniqueName: \"kubernetes.io/projected/df26b3f9-5b87-4125-aa50-7870a0fd2caa-kube-api-access-cgk2t\") pod \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") "
Sep 10 14:52:31.291185 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.291169    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32927f99-9dad-44e8-8199-f65afb0fcc39-plugins-config-volume\") pod \"32927f99-9dad-44e8-8199-f65afb0fcc39\" (UID: \"32927f99-9dad-44e8-8199-f65afb0fcc39\") "
Sep 10 14:52:31.291405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.291217    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-metrics-certs\") pod \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\" (UID: \"df26b3f9-5b87-4125-aa50-7870a0fd2caa\") "
Sep 10 14:52:31.291405 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.291379    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df26b3f9-5b87-4125-aa50-7870a0fd2caa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df26b3f9-5b87-4125-aa50-7870a0fd2caa" (UID: "df26b3f9-5b87-4125-aa50-7870a0fd2caa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:52:31.291519 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.291434    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346aed8f-f0dc-44ab-93f4-1dec902cad72-plugins-config-volume\") pod \"346aed8f-f0dc-44ab-93f4-1dec902cad72\" (UID: \"346aed8f-f0dc-44ab-93f4-1dec902cad72\") "
Sep 10 14:52:31.291987 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.291773    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32927f99-9dad-44e8-8199-f65afb0fcc39-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32927f99-9dad-44e8-8199-f65afb0fcc39" (UID: "32927f99-9dad-44e8-8199-f65afb0fcc39"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:52:31.292180 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.292056    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df26b3f9-5b87-4125-aa50-7870a0fd2caa-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:31.292296 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.292197    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/346aed8f-f0dc-44ab-93f4-1dec902cad72-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "346aed8f-f0dc-44ab-93f4-1dec902cad72" (UID: "346aed8f-f0dc-44ab-93f4-1dec902cad72"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:52:31.293417 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.293377    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/346aed8f-f0dc-44ab-93f4-1dec902cad72-kube-api-access-7wsvb" (OuterVolumeSpecName: "kube-api-access-7wsvb") pod "346aed8f-f0dc-44ab-93f4-1dec902cad72" (UID: "346aed8f-f0dc-44ab-93f4-1dec902cad72"). InnerVolumeSpecName "kube-api-access-7wsvb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:52:31.293796 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.293773    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "df26b3f9-5b87-4125-aa50-7870a0fd2caa" (UID: "df26b3f9-5b87-4125-aa50-7870a0fd2caa"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:52:31.293870 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.293796    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32927f99-9dad-44e8-8199-f65afb0fcc39-kube-api-access-plxpb" (OuterVolumeSpecName: "kube-api-access-plxpb") pod "32927f99-9dad-44e8-8199-f65afb0fcc39" (UID: "32927f99-9dad-44e8-8199-f65afb0fcc39"). InnerVolumeSpecName "kube-api-access-plxpb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:52:31.293990 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.293971    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df26b3f9-5b87-4125-aa50-7870a0fd2caa" (UID: "df26b3f9-5b87-4125-aa50-7870a0fd2caa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:52:31.294140 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.294123    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df26b3f9-5b87-4125-aa50-7870a0fd2caa-kube-api-access-cgk2t" (OuterVolumeSpecName: "kube-api-access-cgk2t") pod "df26b3f9-5b87-4125-aa50-7870a0fd2caa" (UID: "df26b3f9-5b87-4125-aa50-7870a0fd2caa"). InnerVolumeSpecName "kube-api-access-cgk2t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:52:31.393227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.393171    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:31.393227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.393194    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-plxpb\" (UniqueName: \"kubernetes.io/projected/32927f99-9dad-44e8-8199-f65afb0fcc39-kube-api-access-plxpb\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:31.393227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.393203    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wsvb\" (UniqueName: \"kubernetes.io/projected/346aed8f-f0dc-44ab-93f4-1dec902cad72-kube-api-access-7wsvb\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:31.393227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.393212    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgk2t\" (UniqueName: \"kubernetes.io/projected/df26b3f9-5b87-4125-aa50-7870a0fd2caa-kube-api-access-cgk2t\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:31.393227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.393220    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32927f99-9dad-44e8-8199-f65afb0fcc39-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:31.393227 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.393228    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df26b3f9-5b87-4125-aa50-7870a0fd2caa-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:31.393473 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.393236    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/346aed8f-f0dc-44ab-93f4-1dec902cad72-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:31.398349 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.398323    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:31.798476 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.798407    2578 generic.go:356] "Generic (PLEG): container finished" podID="32927f99-9dad-44e8-8199-f65afb0fcc39" containerID="ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7" exitCode=0
Sep 10 14:52:31.798476 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.798454    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"
Sep 10 14:52:31.798643 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.798489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9" event={"ID":"32927f99-9dad-44e8-8199-f65afb0fcc39","Type":"ContainerDied","Data":"ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7"}
Sep 10 14:52:31.798643 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.798539    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9" event={"ID":"32927f99-9dad-44e8-8199-f65afb0fcc39","Type":"ContainerDied","Data":"5d4ff360e816a67f0cf90077d6d5c829492634f57873d73f0f09dc331cc3e8ac"}
Sep 10 14:52:31.798643 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.798563    2578 scope.go:117] "RemoveContainer" containerID="ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7"
Sep 10 14:52:31.799908 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.799880    2578 generic.go:356] "Generic (PLEG): container finished" podID="df26b3f9-5b87-4125-aa50-7870a0fd2caa" containerID="353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb" exitCode=0
Sep 10 14:52:31.800047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.799911    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh" event={"ID":"df26b3f9-5b87-4125-aa50-7870a0fd2caa","Type":"ContainerDied","Data":"353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb"}
Sep 10 14:52:31.800047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.799939    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"
Sep 10 14:52:31.800047 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.799957    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh" event={"ID":"df26b3f9-5b87-4125-aa50-7870a0fd2caa","Type":"ContainerDied","Data":"7d5a83101681bcdc822a2995be84050d4927e42387e2dc886c9ff90e083d996a"}
Sep 10 14:52:31.801456 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.801431    2578 generic.go:356] "Generic (PLEG): container finished" podID="346aed8f-f0dc-44ab-93f4-1dec902cad72" containerID="3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d" exitCode=0
Sep 10 14:52:31.801572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.801458    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"
Sep 10 14:52:31.801572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.801523    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48" event={"ID":"346aed8f-f0dc-44ab-93f4-1dec902cad72","Type":"ContainerDied","Data":"3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d"}
Sep 10 14:52:31.801572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.801563    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48" event={"ID":"346aed8f-f0dc-44ab-93f4-1dec902cad72","Type":"ContainerDied","Data":"a8bdcab491ecd5cdfc9055c85206239d89a20fb9bc2f04e4ab2f690652e6c8b7"}
Sep 10 14:52:31.809240 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.809211    2578 scope.go:117] "RemoveContainer" containerID="ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7"
Sep 10 14:52:31.809537 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:52:31.809517    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7\": container with ID starting with ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7 not found: ID does not exist" containerID="ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7"
Sep 10 14:52:31.809619 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.809549    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7"} err="failed to get container status \"ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7\": rpc error: code = NotFound desc = could not find container \"ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7\": container with ID starting with ea88e9b42954e57f1b7e702a6f82898c5f2117297378e62dcfde24a18e19e5c7 not found: ID does not exist"
Sep 10 14:52:31.809619 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.809574    2578 scope.go:117] "RemoveContainer" containerID="353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb"
Sep 10 14:52:31.817516 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.817498    2578 scope.go:117] "RemoveContainer" containerID="353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb"
Sep 10 14:52:31.817806 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:52:31.817778    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb\": container with ID starting with 353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb not found: ID does not exist" containerID="353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb"
Sep 10 14:52:31.817852 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.817814    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb"} err="failed to get container status \"353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb\": rpc error: code = NotFound desc = could not find container \"353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb\": container with ID starting with 353740a55db7d07cce2adc7c689e5e0459328dc767eabe4b7b70e6e0d6eb3dfb not found: ID does not exist"
Sep 10 14:52:31.817852 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.817828    2578 scope.go:117] "RemoveContainer" containerID="3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d"
Sep 10 14:52:31.823876 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.823855    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"]
Sep 10 14:52:31.829079 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.828884    2578 scope.go:117] "RemoveContainer" containerID="3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d"
Sep 10 14:52:31.829204 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:52:31.829181    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d\": container with ID starting with 3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d not found: ID does not exist" containerID="3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d"
Sep 10 14:52:31.829256 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.829213    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d"} err="failed to get container status \"3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d\": rpc error: code = NotFound desc = could not find container \"3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d\": container with ID starting with 3e21d10df605b75c766b6b8f00867266026c1531e7cc7446c98a74e1e513a93d not found: ID does not exist"
Sep 10 14:52:31.829296 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.829261    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-3876ff-dbcf68c6b-f2ql9"]
Sep 10 14:52:31.839416 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.839376    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"]
Sep 10 14:52:31.841335 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.841317    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-3876ff-79b4f9d98c-b6c48"]
Sep 10 14:52:31.849438 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.849419    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"]
Sep 10 14:52:31.853406 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:31.853388    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-3876ff-5b5cc65d98-jd5rh"]
Sep 10 14:52:32.725171 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:32.725137    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32927f99-9dad-44e8-8199-f65afb0fcc39" path="/var/lib/kubelet/pods/32927f99-9dad-44e8-8199-f65afb0fcc39/volumes"
Sep 10 14:52:32.725556 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:32.725534    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="346aed8f-f0dc-44ab-93f4-1dec902cad72" path="/var/lib/kubelet/pods/346aed8f-f0dc-44ab-93f4-1dec902cad72/volumes"
Sep 10 14:52:32.725974 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:32.725953    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df26b3f9-5b87-4125-aa50-7870a0fd2caa" path="/var/lib/kubelet/pods/df26b3f9-5b87-4125-aa50-7870a0fd2caa/volumes"
Sep 10 14:52:37.263557 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:37.263506    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:37.362836 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:37.362798    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:37.512215 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:37.512174    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:41.086539 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:41.086495    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:56.629447 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.629412    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:56.666297 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.666259    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"]
Sep 10 14:52:56.667100 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.667054    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt" podUID="5a514fd2-e9e3-4148-8df9-ba560707c9ef" containerName="payload-pre-processing" containerID="cri-o://f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e" gracePeriod=30
Sep 10 14:52:56.727004 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.726965    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:56.826642 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.826608    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:56.908944 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.908924    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:52:56.927497 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.927465    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:56.931431 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.931404    2578 generic.go:356] "Generic (PLEG): container finished" podID="5a514fd2-e9e3-4148-8df9-ba560707c9ef" containerID="f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e" exitCode=0
Sep 10 14:52:56.931534 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.931464    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"
Sep 10 14:52:56.931534 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.931489    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt" event={"ID":"5a514fd2-e9e3-4148-8df9-ba560707c9ef","Type":"ContainerDied","Data":"f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e"}
Sep 10 14:52:56.931672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.931551    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt" event={"ID":"5a514fd2-e9e3-4148-8df9-ba560707c9ef","Type":"ContainerDied","Data":"200cef9bf16048170318dbc841503bd8ecd220378a2252cf720d7f7d723fe4ef"}
Sep 10 14:52:56.931672 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.931585    2578 scope.go:117] "RemoveContainer" containerID="f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e"
Sep 10 14:52:56.941642 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.941623    2578 scope.go:117] "RemoveContainer" containerID="f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e"
Sep 10 14:52:56.941986 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:52:56.941967    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e\": container with ID starting with f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e not found: ID does not exist" containerID="f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e"
Sep 10 14:52:56.942043 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.941994    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e"} err="failed to get container status \"f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e\": rpc error: code = NotFound desc = could not find container \"f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e\": container with ID starting with f785b405810828812e4eddbc206c07dae254ef1ace42ea53f920ca738dcaa91e not found: ID does not exist"
Sep 10 14:52:56.989571 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.989553    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h5s5k\" (UniqueName: \"kubernetes.io/projected/5a514fd2-e9e3-4148-8df9-ba560707c9ef-kube-api-access-h5s5k\") pod \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\" (UID: \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\") "
Sep 10 14:52:56.989683 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.989649    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a514fd2-e9e3-4148-8df9-ba560707c9ef-plugins-config-volume\") pod \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\" (UID: \"5a514fd2-e9e3-4148-8df9-ba560707c9ef\") "
Sep 10 14:52:56.990006 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.989982    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a514fd2-e9e3-4148-8df9-ba560707c9ef-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5a514fd2-e9e3-4148-8df9-ba560707c9ef" (UID: "5a514fd2-e9e3-4148-8df9-ba560707c9ef"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:52:56.991677 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:56.991658    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a514fd2-e9e3-4148-8df9-ba560707c9ef-kube-api-access-h5s5k" (OuterVolumeSpecName: "kube-api-access-h5s5k") pod "5a514fd2-e9e3-4148-8df9-ba560707c9ef" (UID: "5a514fd2-e9e3-4148-8df9-ba560707c9ef"). InnerVolumeSpecName "kube-api-access-h5s5k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:52:57.091131 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:57.091109    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h5s5k\" (UniqueName: \"kubernetes.io/projected/5a514fd2-e9e3-4148-8df9-ba560707c9ef-kube-api-access-h5s5k\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:57.091131 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:57.091132    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5a514fd2-e9e3-4148-8df9-ba560707c9ef-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:52:57.188708 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:57.188656    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:52:57.251924 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:57.251898    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"]
Sep 10 14:52:57.253679 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:57.253661    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-cc50fd-dd866dc66-9mqzt"]
Sep 10 14:52:58.724761 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:52:58.724715    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a514fd2-e9e3-4148-8df9-ba560707c9ef" path="/var/lib/kubelet/pods/5a514fd2-e9e3-4148-8df9-ba560707c9ef/volumes"
Sep 10 14:53:02.216253 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:02.216214    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:38.713570 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:38.713533    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:39.259762 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:39.259727    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:44.742641 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:44.742604    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:45.758567 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:45.758528    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:58.889668 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:58.889629    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:58.989652 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:58.989614    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:59.180547 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:59.180452    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:53:59.658268 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:53:59.658228    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:10.076603 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.076569    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"]
Sep 10 14:54:10.077271 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.076811    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd" podUID="621970f6-4aaf-4d46-ae90-a713bb7e6daf" containerName="maas-api" containerID="cri-o://d28af495794df55e96c568cf90003c7ec988acfbdc168d0ccc6ec20aa724c567" gracePeriod=30
Sep 10 14:54:10.104637 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.104566    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:10.175406 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.175377    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:10.238414 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.238369    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:10.300814 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.300779    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:10.307568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.307543    2578 generic.go:356] "Generic (PLEG): container finished" podID="621970f6-4aaf-4d46-ae90-a713bb7e6daf" containerID="d28af495794df55e96c568cf90003c7ec988acfbdc168d0ccc6ec20aa724c567" exitCode=0
Sep 10 14:54:10.307688 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.307609    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd" event={"ID":"621970f6-4aaf-4d46-ae90-a713bb7e6daf","Type":"ContainerDied","Data":"d28af495794df55e96c568cf90003c7ec988acfbdc168d0ccc6ec20aa724c567"}
Sep 10 14:54:10.326573 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.326553    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:54:10.374130 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.374070    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:10.450890 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.450859    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls\") pod \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") "
Sep 10 14:54:10.451034 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.450914    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xtq9p\" (UniqueName: \"kubernetes.io/projected/621970f6-4aaf-4d46-ae90-a713bb7e6daf-kube-api-access-xtq9p\") pod \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") "
Sep 10 14:54:10.451034 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.451024    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-metrics-certs\") pod \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") "
Sep 10 14:54:10.451144 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.451080    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621970f6-4aaf-4d46-ae90-a713bb7e6daf-openshift-service-ca\") pod \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\" (UID: \"621970f6-4aaf-4d46-ae90-a713bb7e6daf\") "
Sep 10 14:54:10.451466 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.451434    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/621970f6-4aaf-4d46-ae90-a713bb7e6daf-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "621970f6-4aaf-4d46-ae90-a713bb7e6daf" (UID: "621970f6-4aaf-4d46-ae90-a713bb7e6daf"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:54:10.453158 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.453126    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "621970f6-4aaf-4d46-ae90-a713bb7e6daf" (UID: "621970f6-4aaf-4d46-ae90-a713bb7e6daf"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:54:10.453245 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.453188    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "621970f6-4aaf-4d46-ae90-a713bb7e6daf" (UID: "621970f6-4aaf-4d46-ae90-a713bb7e6daf"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:54:10.453245 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.453189    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/621970f6-4aaf-4d46-ae90-a713bb7e6daf-kube-api-access-xtq9p" (OuterVolumeSpecName: "kube-api-access-xtq9p") pod "621970f6-4aaf-4d46-ae90-a713bb7e6daf" (UID: "621970f6-4aaf-4d46-ae90-a713bb7e6daf"). InnerVolumeSpecName "kube-api-access-xtq9p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:54:10.552336 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.552311    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:10.552336 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.552336    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/621970f6-4aaf-4d46-ae90-a713bb7e6daf-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:10.552336 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.552346    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/621970f6-4aaf-4d46-ae90-a713bb7e6daf-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:10.552532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.552354    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xtq9p\" (UniqueName: \"kubernetes.io/projected/621970f6-4aaf-4d46-ae90-a713bb7e6daf-kube-api-access-xtq9p\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:10.730626 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:10.730546    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:11.314198 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:11.314171    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"
Sep 10 14:54:11.314597 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:11.314209    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd" event={"ID":"621970f6-4aaf-4d46-ae90-a713bb7e6daf","Type":"ContainerDied","Data":"cb5b6bbb1e3729b46c200cede88555f77d50f5b10b66935a829f43b8f08e5291"}
Sep 10 14:54:11.314597 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:11.314284    2578 scope.go:117] "RemoveContainer" containerID="d28af495794df55e96c568cf90003c7ec988acfbdc168d0ccc6ec20aa724c567"
Sep 10 14:54:11.332258 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:11.332233    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"]
Sep 10 14:54:11.337501 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:11.337479    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-eaf326-54fb7989df-dcjqd"]
Sep 10 14:54:12.724806 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:12.724775    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="621970f6-4aaf-4d46-ae90-a713bb7e6daf" path="/var/lib/kubelet/pods/621970f6-4aaf-4d46-ae90-a713bb7e6daf/volumes"
Sep 10 14:54:13.844738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:13.844702    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:16.417197 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:16.417105    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:16.479555 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:16.479525    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:16.757352 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:16.757266    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:17.154448 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:17.154415    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:27.496014 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.495979    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"]
Sep 10 14:54:27.496478 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.496216    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh" podUID="b5907479-c0bb-4952-8e9e-a8bc23331d79" containerName="maas-api" containerID="cri-o://652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2" gracePeriod=30
Sep 10 14:54:27.511042 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.511010    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:27.546815 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.546790    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"]
Sep 10 14:54:27.547438 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.547406    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7" podUID="59f74e3b-24b1-49d0-b3b4-110c75cfc65d" containerName="payload-processing" containerID="cri-o://0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8" gracePeriod=30
Sep 10 14:54:27.554952 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.554924    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"]
Sep 10 14:54:27.555203 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.555156    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk" podUID="c1da6868-fed8-4751-8e1c-d2444892407a" containerName="payload-pre-processing" containerID="cri-o://da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191" gracePeriod=30
Sep 10 14:54:27.593643 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.593597    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:27.660085 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.660056    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:27.728348 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.728318    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:27.844184 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.844157    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:54:27.847159 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.847124    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:54:27.851893 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.851868    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:54:27.892871 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.892848    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7ndmk\" (UniqueName: \"kubernetes.io/projected/c1da6868-fed8-4751-8e1c-d2444892407a-kube-api-access-7ndmk\") pod \"c1da6868-fed8-4751-8e1c-d2444892407a\" (UID: \"c1da6868-fed8-4751-8e1c-d2444892407a\") "
Sep 10 14:54:27.893002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.892890    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-plugins-config-volume\") pod \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\" (UID: \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\") "
Sep 10 14:54:27.893002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.892909    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5907479-c0bb-4952-8e9e-a8bc23331d79-openshift-service-ca\") pod \"b5907479-c0bb-4952-8e9e-a8bc23331d79\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") "
Sep 10 14:54:27.893002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.892954    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wddqd\" (UniqueName: \"kubernetes.io/projected/b5907479-c0bb-4952-8e9e-a8bc23331d79-kube-api-access-wddqd\") pod \"b5907479-c0bb-4952-8e9e-a8bc23331d79\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") "
Sep 10 14:54:27.893002 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.892978    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1da6868-fed8-4751-8e1c-d2444892407a-plugins-config-volume\") pod \"c1da6868-fed8-4751-8e1c-d2444892407a\" (UID: \"c1da6868-fed8-4751-8e1c-d2444892407a\") "
Sep 10 14:54:27.893276 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893004    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-metrics-certs\") pod \"b5907479-c0bb-4952-8e9e-a8bc23331d79\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") "
Sep 10 14:54:27.893276 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893030    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwrp4\" (UniqueName: \"kubernetes.io/projected/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-kube-api-access-vwrp4\") pod \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\" (UID: \"59f74e3b-24b1-49d0-b3b4-110c75cfc65d\") "
Sep 10 14:54:27.893276 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893083    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-maas-api-tls\") pod \"b5907479-c0bb-4952-8e9e-a8bc23331d79\" (UID: \"b5907479-c0bb-4952-8e9e-a8bc23331d79\") "
Sep 10 14:54:27.893493 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893293    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "59f74e3b-24b1-49d0-b3b4-110c75cfc65d" (UID: "59f74e3b-24b1-49d0-b3b4-110c75cfc65d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:54:27.893493 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893311    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5907479-c0bb-4952-8e9e-a8bc23331d79-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5907479-c0bb-4952-8e9e-a8bc23331d79" (UID: "b5907479-c0bb-4952-8e9e-a8bc23331d79"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:54:27.893493 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893324    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c1da6868-fed8-4751-8e1c-d2444892407a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c1da6868-fed8-4751-8e1c-d2444892407a" (UID: "c1da6868-fed8-4751-8e1c-d2444892407a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 14:54:27.894367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893563    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:27.894367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893585    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5907479-c0bb-4952-8e9e-a8bc23331d79-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:27.894367 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.893599    2578 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c1da6868-fed8-4751-8e1c-d2444892407a-plugins-config-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:27.896879 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.896851    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5907479-c0bb-4952-8e9e-a8bc23331d79-kube-api-access-wddqd" (OuterVolumeSpecName: "kube-api-access-wddqd") pod "b5907479-c0bb-4952-8e9e-a8bc23331d79" (UID: "b5907479-c0bb-4952-8e9e-a8bc23331d79"). InnerVolumeSpecName "kube-api-access-wddqd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:54:27.896879 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.896828    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1da6868-fed8-4751-8e1c-d2444892407a-kube-api-access-7ndmk" (OuterVolumeSpecName: "kube-api-access-7ndmk") pod "c1da6868-fed8-4751-8e1c-d2444892407a" (UID: "c1da6868-fed8-4751-8e1c-d2444892407a"). InnerVolumeSpecName "kube-api-access-7ndmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:54:27.897771 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.897717    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-kube-api-access-vwrp4" (OuterVolumeSpecName: "kube-api-access-vwrp4") pod "59f74e3b-24b1-49d0-b3b4-110c75cfc65d" (UID: "59f74e3b-24b1-49d0-b3b4-110c75cfc65d"). InnerVolumeSpecName "kube-api-access-vwrp4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:54:27.904055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.900033    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5907479-c0bb-4952-8e9e-a8bc23331d79" (UID: "b5907479-c0bb-4952-8e9e-a8bc23331d79"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:54:27.904055 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.901883    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b5907479-c0bb-4952-8e9e-a8bc23331d79" (UID: "b5907479-c0bb-4952-8e9e-a8bc23331d79"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:54:27.994441 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.994415    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wddqd\" (UniqueName: \"kubernetes.io/projected/b5907479-c0bb-4952-8e9e-a8bc23331d79-kube-api-access-wddqd\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:27.994441 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.994441    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:27.994578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.994452    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwrp4\" (UniqueName: \"kubernetes.io/projected/59f74e3b-24b1-49d0-b3b4-110c75cfc65d-kube-api-access-vwrp4\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:27.994578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.994461    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5907479-c0bb-4952-8e9e-a8bc23331d79-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:27.994578 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:27.994470    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7ndmk\" (UniqueName: \"kubernetes.io/projected/c1da6868-fed8-4751-8e1c-d2444892407a-kube-api-access-7ndmk\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:28.111181 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.111153    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:28.401081 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.401008    2578 generic.go:356] "Generic (PLEG): container finished" podID="b5907479-c0bb-4952-8e9e-a8bc23331d79" containerID="652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2" exitCode=0
Sep 10 14:54:28.401081 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.401070    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"
Sep 10 14:54:28.401265 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.401094    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh" event={"ID":"b5907479-c0bb-4952-8e9e-a8bc23331d79","Type":"ContainerDied","Data":"652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2"}
Sep 10 14:54:28.401265 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.401187    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh" event={"ID":"b5907479-c0bb-4952-8e9e-a8bc23331d79","Type":"ContainerDied","Data":"896c0c735022febf0926dfd6dca3d6ffbe9aeec5aca1e5d0c9ad0086db57b0e7"}
Sep 10 14:54:28.401265 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.401215    2578 scope.go:117] "RemoveContainer" containerID="652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2"
Sep 10 14:54:28.402542 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.402516    2578 generic.go:356] "Generic (PLEG): container finished" podID="c1da6868-fed8-4751-8e1c-d2444892407a" containerID="da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191" exitCode=0
Sep 10 14:54:28.402622 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.402583    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"
Sep 10 14:54:28.402668 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.402613    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk" event={"ID":"c1da6868-fed8-4751-8e1c-d2444892407a","Type":"ContainerDied","Data":"da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191"}
Sep 10 14:54:28.402668 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.402662    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk" event={"ID":"c1da6868-fed8-4751-8e1c-d2444892407a","Type":"ContainerDied","Data":"c739886f7725a424be42db3378167e3414e9e412167a0c8b609bcf29e4f2db59"}
Sep 10 14:54:28.404178 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.404151    2578 generic.go:356] "Generic (PLEG): container finished" podID="59f74e3b-24b1-49d0-b3b4-110c75cfc65d" containerID="0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8" exitCode=0
Sep 10 14:54:28.404326 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.404205    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7" event={"ID":"59f74e3b-24b1-49d0-b3b4-110c75cfc65d","Type":"ContainerDied","Data":"0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8"}
Sep 10 14:54:28.404326 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.404218    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"
Sep 10 14:54:28.404326 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.404244    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7" event={"ID":"59f74e3b-24b1-49d0-b3b4-110c75cfc65d","Type":"ContainerDied","Data":"78a19ae0d1f1d6ccac81e04a09ad3783a5a90e953de14f91ddc28f915cfca21e"}
Sep 10 14:54:28.411376 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.411361    2578 scope.go:117] "RemoveContainer" containerID="652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2"
Sep 10 14:54:28.411645 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:54:28.411626    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2\": container with ID starting with 652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2 not found: ID does not exist" containerID="652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2"
Sep 10 14:54:28.411715 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.411652    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2"} err="failed to get container status \"652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2\": rpc error: code = NotFound desc = could not find container \"652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2\": container with ID starting with 652098508ea5ea95faead980d40136f4a4226a61456a78a7ea4aaefb749acdf2 not found: ID does not exist"
Sep 10 14:54:28.411715 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.411669    2578 scope.go:117] "RemoveContainer" containerID="da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191"
Sep 10 14:54:28.419926 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.419909    2578 scope.go:117] "RemoveContainer" containerID="da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191"
Sep 10 14:54:28.420176 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:54:28.420158    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191\": container with ID starting with da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191 not found: ID does not exist" containerID="da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191"
Sep 10 14:54:28.420224 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.420183    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191"} err="failed to get container status \"da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191\": rpc error: code = NotFound desc = could not find container \"da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191\": container with ID starting with da9f2309f723431590db6a9787cb8af1843bbee8a39e9edff185f6dce6a84191 not found: ID does not exist"
Sep 10 14:54:28.420224 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.420200    2578 scope.go:117] "RemoveContainer" containerID="0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8"
Sep 10 14:54:28.423811 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.423789    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"]
Sep 10 14:54:28.428984 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.428968    2578 scope.go:117] "RemoveContainer" containerID="0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8"
Sep 10 14:54:28.429279 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:54:28.429260    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8\": container with ID starting with 0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8 not found: ID does not exist" containerID="0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8"
Sep 10 14:54:28.429338 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.429281    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w3-85d86b-74b49b6c95-dq9wh"]
Sep 10 14:54:28.429338 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.429286    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8"} err="failed to get container status \"0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8\": rpc error: code = NotFound desc = could not find container \"0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8\": container with ID starting with 0ea34afb34c9f31e98aa627b321cef5102bad3519663eb7ecb7338732ad530f8 not found: ID does not exist"
Sep 10 14:54:28.435771 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.435736    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"]
Sep 10 14:54:28.439813 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.439793    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w3-85d86b-55574f977c-bfkm7"]
Sep 10 14:54:28.448293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.448275    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"]
Sep 10 14:54:28.452277 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.452258    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-85d86b-7964fb96c4-5kdhk"]
Sep 10 14:54:28.725105 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.725026    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="59f74e3b-24b1-49d0-b3b4-110c75cfc65d" path="/var/lib/kubelet/pods/59f74e3b-24b1-49d0-b3b4-110c75cfc65d/volumes"
Sep 10 14:54:28.725584 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.725555    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5907479-c0bb-4952-8e9e-a8bc23331d79" path="/var/lib/kubelet/pods/b5907479-c0bb-4952-8e9e-a8bc23331d79/volumes"
Sep 10 14:54:28.725949 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:28.725931    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1da6868-fed8-4751-8e1c-d2444892407a" path="/var/lib/kubelet/pods/c1da6868-fed8-4751-8e1c-d2444892407a/volumes"
Sep 10 14:54:34.097900 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:34.097854    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:34.155511 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:34.155483    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:34.375724 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:34.375631    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:34.940480 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:34.940449    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:45.245329 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:45.245291    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:45.319064 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:45.319022    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:45.382693 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:45.382654    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:45.455310 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:45.455273    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:45.537543 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:45.537464    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:45.823538 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:45.823500    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:51.893414 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:51.893380    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:51.994129 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:51.994101    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:52.156006 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:52.155901    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:54:54.837785 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:54.837727    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-rzw7w"]
Sep 10 14:54:54.838169 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:54.837973    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55b7c7d477-rzw7w" podUID="a7e52a2b-6e5e-4f5f-bc86-904520f300ee" containerName="manager" containerID="cri-o://fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516" gracePeriod=30
Sep 10 14:54:55.091955 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.091898    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:54:55.208338 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.208304    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs\") pod \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") "
Sep 10 14:54:55.208517 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.208379    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-webhook-cert\") pod \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") "
Sep 10 14:54:55.208517 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.208399    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmx2k\" (UniqueName: \"kubernetes.io/projected/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-kube-api-access-bmx2k\") pod \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\" (UID: \"a7e52a2b-6e5e-4f5f-bc86-904520f300ee\") "
Sep 10 14:54:55.210475 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.210446    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "a7e52a2b-6e5e-4f5f-bc86-904520f300ee" (UID: "a7e52a2b-6e5e-4f5f-bc86-904520f300ee"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:54:55.210568 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.210516    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "a7e52a2b-6e5e-4f5f-bc86-904520f300ee" (UID: "a7e52a2b-6e5e-4f5f-bc86-904520f300ee"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:54:55.210632 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.210612    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-kube-api-access-bmx2k" (OuterVolumeSpecName: "kube-api-access-bmx2k") pod "a7e52a2b-6e5e-4f5f-bc86-904520f300ee" (UID: "a7e52a2b-6e5e-4f5f-bc86-904520f300ee"). InnerVolumeSpecName "kube-api-access-bmx2k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:54:55.309521 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.309495    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:55.309521 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.309517    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-webhook-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:55.309659 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.309526    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmx2k\" (UniqueName: \"kubernetes.io/projected/a7e52a2b-6e5e-4f5f-bc86-904520f300ee-kube-api-access-bmx2k\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:54:55.554444 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.554367    2578 generic.go:356] "Generic (PLEG): container finished" podID="a7e52a2b-6e5e-4f5f-bc86-904520f300ee" containerID="fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516" exitCode=0
Sep 10 14:54:55.554444 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.554410    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-rzw7w"
Sep 10 14:54:55.554444 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.554426    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-rzw7w" event={"ID":"a7e52a2b-6e5e-4f5f-bc86-904520f300ee","Type":"ContainerDied","Data":"fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516"}
Sep 10 14:54:55.554690 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.554479    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-rzw7w" event={"ID":"a7e52a2b-6e5e-4f5f-bc86-904520f300ee","Type":"ContainerDied","Data":"1e3ef7a8d07d2bacd13616f219db04130bce78a7e7021ba6b76992c17647069d"}
Sep 10 14:54:55.554690 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.554508    2578 scope.go:117] "RemoveContainer" containerID="fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516"
Sep 10 14:54:55.564697 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.564680    2578 scope.go:117] "RemoveContainer" containerID="fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516"
Sep 10 14:54:55.565013 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:54:55.564991    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516\": container with ID starting with fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516 not found: ID does not exist" containerID="fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516"
Sep 10 14:54:55.565086 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.565024    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516"} err="failed to get container status \"fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516\": rpc error: code = NotFound desc = could not find container \"fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516\": container with ID starting with fc5729c1d876cb7725c8803589ff4e0b332ddd2b78c53c69499859eacfb07516 not found: ID does not exist"
Sep 10 14:54:55.575036 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.575014    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-rzw7w"]
Sep 10 14:54:55.579465 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:55.579441    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-rzw7w"]
Sep 10 14:54:56.724282 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:54:56.724248    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7e52a2b-6e5e-4f5f-bc86-904520f300ee" path="/var/lib/kubelet/pods/a7e52a2b-6e5e-4f5f-bc86-904520f300ee/volumes"
Sep 10 14:55:17.667661 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:17.667622    2578 generic.go:356] "Generic (PLEG): container finished" podID="37d62584-d7e1-4aed-9fcd-37ffbeaa687a" containerID="98768608e9da0ed90b917a6a17f6325aecbc75a12468e91aa1b840f12304eb91" exitCode=1
Sep 10 14:55:17.668074 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:17.667668    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4" event={"ID":"37d62584-d7e1-4aed-9fcd-37ffbeaa687a","Type":"ContainerDied","Data":"98768608e9da0ed90b917a6a17f6325aecbc75a12468e91aa1b840f12304eb91"}
Sep 10 14:55:17.668074 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:17.667967    2578 scope.go:117] "RemoveContainer" containerID="98768608e9da0ed90b917a6a17f6325aecbc75a12468e91aa1b840f12304eb91"
Sep 10 14:55:18.674693 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:18.674660    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4" event={"ID":"37d62584-d7e1-4aed-9fcd-37ffbeaa687a","Type":"ContainerStarted","Data":"2970196cfe15e2515de6dbe605ed5c84f53a8d5137f183d9110139b1bf00d3c8"}
Sep 10 14:55:18.675111 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:18.674889    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:55:20.480244 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:20.480187    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:55:49.681780 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:49.681666    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-d2fr4"
Sep 10 14:55:56.378053 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378020    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55b7c7d477-kxzh6"]
Sep 10 14:55:56.378436 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378412    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="621970f6-4aaf-4d46-ae90-a713bb7e6daf" containerName="maas-api"
Sep 10 14:55:56.378436 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378424    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="621970f6-4aaf-4d46-ae90-a713bb7e6daf" containerName="maas-api"
Sep 10 14:55:56.378436 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378438    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1da6868-fed8-4751-8e1c-d2444892407a" containerName="payload-pre-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378444    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1da6868-fed8-4751-8e1c-d2444892407a" containerName="payload-pre-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378450    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df26b3f9-5b87-4125-aa50-7870a0fd2caa" containerName="maas-api"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378456    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="df26b3f9-5b87-4125-aa50-7870a0fd2caa" containerName="maas-api"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378467    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32927f99-9dad-44e8-8199-f65afb0fcc39" containerName="payload-pre-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378472    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="32927f99-9dad-44e8-8199-f65afb0fcc39" containerName="payload-pre-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378479    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5907479-c0bb-4952-8e9e-a8bc23331d79" containerName="maas-api"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378484    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5907479-c0bb-4952-8e9e-a8bc23331d79" containerName="maas-api"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378490    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="59f74e3b-24b1-49d0-b3b4-110c75cfc65d" containerName="payload-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378496    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="59f74e3b-24b1-49d0-b3b4-110c75cfc65d" containerName="payload-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378501    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a514fd2-e9e3-4148-8df9-ba560707c9ef" containerName="payload-pre-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378508    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a514fd2-e9e3-4148-8df9-ba560707c9ef" containerName="payload-pre-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378513    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="346aed8f-f0dc-44ab-93f4-1dec902cad72" containerName="payload-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378520    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="346aed8f-f0dc-44ab-93f4-1dec902cad72" containerName="payload-processing"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378531    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7e52a2b-6e5e-4f5f-bc86-904520f300ee" containerName="manager"
Sep 10 14:55:56.378544 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378537    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7e52a2b-6e5e-4f5f-bc86-904520f300ee" containerName="manager"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378588    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7e52a2b-6e5e-4f5f-bc86-904520f300ee" containerName="manager"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378597    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a514fd2-e9e3-4148-8df9-ba560707c9ef" containerName="payload-pre-processing"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378603    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5907479-c0bb-4952-8e9e-a8bc23331d79" containerName="maas-api"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378608    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c1da6868-fed8-4751-8e1c-d2444892407a" containerName="payload-pre-processing"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378615    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="32927f99-9dad-44e8-8199-f65afb0fcc39" containerName="payload-pre-processing"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378621    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="df26b3f9-5b87-4125-aa50-7870a0fd2caa" containerName="maas-api"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378627    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="59f74e3b-24b1-49d0-b3b4-110c75cfc65d" containerName="payload-processing"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378631    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="346aed8f-f0dc-44ab-93f4-1dec902cad72" containerName="payload-processing"
Sep 10 14:55:56.379029 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.378637    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="621970f6-4aaf-4d46-ae90-a713bb7e6daf" containerName="maas-api"
Sep 10 14:55:56.382600 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.382582    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.384903 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.384875    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 14:55:56.384903 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.384889    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 14:55:56.384903 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.384875    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 14:55:56.385156 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.384890    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-754rh\""
Sep 10 14:55:56.393630 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.393603    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-kxzh6"]
Sep 10 14:55:56.511350 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.511320    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6vss\" (UniqueName: \"kubernetes.io/projected/ce84b34a-6340-47f7-8d08-8912fc82f4ce-kube-api-access-r6vss\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.511479 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.511363    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-webhook-cert\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.511479 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.511394    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-metrics-certs\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.612693 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.612664    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-metrics-certs\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.612808 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.612726    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6vss\" (UniqueName: \"kubernetes.io/projected/ce84b34a-6340-47f7-8d08-8912fc82f4ce-kube-api-access-r6vss\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.612808 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.612768    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-webhook-cert\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.615303 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.615272    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-metrics-certs\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.615402 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.615358    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-webhook-cert\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.623248 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.623224    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6vss\" (UniqueName: \"kubernetes.io/projected/ce84b34a-6340-47f7-8d08-8912fc82f4ce-kube-api-access-r6vss\") pod \"maas-controller-55b7c7d477-kxzh6\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") " pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.696016 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.695963    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:56.967041 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:56.966970    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-kxzh6"]
Sep 10 14:55:56.969256 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:55:56.969230    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce84b34a_6340_47f7_8d08_8912fc82f4ce.slice/crio-495cbf5273c4b792587ced2d843f54cf9b896c3157a8b4004213feb74c0dea08 WatchSource:0}: Error finding container 495cbf5273c4b792587ced2d843f54cf9b896c3157a8b4004213feb74c0dea08: Status 404 returned error can't find the container with id 495cbf5273c4b792587ced2d843f54cf9b896c3157a8b4004213feb74c0dea08
Sep 10 14:55:57.863837 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:57.863805    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-kxzh6" event={"ID":"ce84b34a-6340-47f7-8d08-8912fc82f4ce","Type":"ContainerStarted","Data":"be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3"}
Sep 10 14:55:57.864197 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:57.863855    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-kxzh6" event={"ID":"ce84b34a-6340-47f7-8d08-8912fc82f4ce","Type":"ContainerStarted","Data":"495cbf5273c4b792587ced2d843f54cf9b896c3157a8b4004213feb74c0dea08"}
Sep 10 14:55:57.864197 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:57.863916    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:55:57.881381 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:55:57.881337    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55b7c7d477-kxzh6" podStartSLOduration=1.296774142 podStartE2EDuration="1.88132332s" totalImagesPullingTime="584.549178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:55:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:55:57.879587511 +0000 UTC m=+2163.647310305" watchObservedRunningTime="2026-09-10 14:55:57.88132332 +0000 UTC m=+2163.649046101"
Sep 10 14:56:08.871389 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:08.871359    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:56:13.031890 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:13.031842    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:14.427173 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:14.427138    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:15.358959 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:15.358923    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:15.949509 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:15.949478    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:16.049352 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.049316    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-kxzh6"]
Sep 10 14:56:16.049559 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.049513    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-55b7c7d477-kxzh6" podUID="ce84b34a-6340-47f7-8d08-8912fc82f4ce" containerName="manager" containerID="cri-o://be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3" gracePeriod=30
Sep 10 14:56:16.399673 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.399645    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:56:16.585549 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.585507    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-webhook-cert\") pod \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") "
Sep 10 14:56:16.585782 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.585573    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-metrics-certs\") pod \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") "
Sep 10 14:56:16.585782 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.585596    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6vss\" (UniqueName: \"kubernetes.io/projected/ce84b34a-6340-47f7-8d08-8912fc82f4ce-kube-api-access-r6vss\") pod \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\" (UID: \"ce84b34a-6340-47f7-8d08-8912fc82f4ce\") "
Sep 10 14:56:16.587878 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.587833    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ce84b34a-6340-47f7-8d08-8912fc82f4ce" (UID: "ce84b34a-6340-47f7-8d08-8912fc82f4ce"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:16.587991 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.587872    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce84b34a-6340-47f7-8d08-8912fc82f4ce-kube-api-access-r6vss" (OuterVolumeSpecName: "kube-api-access-r6vss") pod "ce84b34a-6340-47f7-8d08-8912fc82f4ce" (UID: "ce84b34a-6340-47f7-8d08-8912fc82f4ce"). InnerVolumeSpecName "kube-api-access-r6vss". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:56:16.587991 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.587896    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "ce84b34a-6340-47f7-8d08-8912fc82f4ce" (UID: "ce84b34a-6340-47f7-8d08-8912fc82f4ce"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 14:56:16.686548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.686517    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:56:16.686548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.686545    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6vss\" (UniqueName: \"kubernetes.io/projected/ce84b34a-6340-47f7-8d08-8912fc82f4ce-kube-api-access-r6vss\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:56:16.686548 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.686555    2578 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ce84b34a-6340-47f7-8d08-8912fc82f4ce-webhook-cert\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:56:16.962686 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.962599    2578 generic.go:356] "Generic (PLEG): container finished" podID="ce84b34a-6340-47f7-8d08-8912fc82f4ce" containerID="be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3" exitCode=0
Sep 10 14:56:16.962686 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.962655    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-kxzh6"
Sep 10 14:56:16.962686 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.962669    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-kxzh6" event={"ID":"ce84b34a-6340-47f7-8d08-8912fc82f4ce","Type":"ContainerDied","Data":"be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3"}
Sep 10 14:56:16.963206 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.962734    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-kxzh6" event={"ID":"ce84b34a-6340-47f7-8d08-8912fc82f4ce","Type":"ContainerDied","Data":"495cbf5273c4b792587ced2d843f54cf9b896c3157a8b4004213feb74c0dea08"}
Sep 10 14:56:16.963206 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.962784    2578 scope.go:117] "RemoveContainer" containerID="be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3"
Sep 10 14:56:16.971262 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.971246    2578 scope.go:117] "RemoveContainer" containerID="be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3"
Sep 10 14:56:16.971541 ip-10-0-142-68 kubenswrapper[2578]: E0910 14:56:16.971519    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3\": container with ID starting with be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3 not found: ID does not exist" containerID="be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3"
Sep 10 14:56:16.971612 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.971547    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3"} err="failed to get container status \"be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3\": rpc error: code = NotFound desc = could not find container \"be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3\": container with ID starting with be783989bc4f7f4fee7248cb0942ce78eb8bf08cd2631146ebd4713ee63899b3 not found: ID does not exist"
Sep 10 14:56:16.979293 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.979273    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-kxzh6"]
Sep 10 14:56:16.982621 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:16.982601    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-kxzh6"]
Sep 10 14:56:17.600640 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.600601    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:17.704071 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.704039    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-55b7c7d477-j6zng"]
Sep 10 14:56:17.704429 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.704416    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ce84b34a-6340-47f7-8d08-8912fc82f4ce" containerName="manager"
Sep 10 14:56:17.704474 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.704431    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="ce84b34a-6340-47f7-8d08-8912fc82f4ce" containerName="manager"
Sep 10 14:56:17.704514 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.704496    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="ce84b34a-6340-47f7-8d08-8912fc82f4ce" containerName="manager"
Sep 10 14:56:17.708738 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.708718    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:17.711332 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.711307    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 10 14:56:17.711332 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.711318    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 10 14:56:17.711521 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.711318    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 10 14:56:17.711521 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.711318    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-754rh\""
Sep 10 14:56:17.715781 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.715735    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-j6zng"]
Sep 10 14:56:17.897837 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.897731    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxm45\" (UniqueName: \"kubernetes.io/projected/419d0a5a-0ed9-445f-9f97-337eb2726370-kube-api-access-kxm45\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:17.897837 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.897810    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/419d0a5a-0ed9-445f-9f97-337eb2726370-webhook-cert\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:17.898026 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.897889    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/419d0a5a-0ed9-445f-9f97-337eb2726370-metrics-certs\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:17.998829 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.998797    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/419d0a5a-0ed9-445f-9f97-337eb2726370-webhook-cert\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:17.999232 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.998870    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/419d0a5a-0ed9-445f-9f97-337eb2726370-metrics-certs\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:17.999232 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:17.998909    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxm45\" (UniqueName: \"kubernetes.io/projected/419d0a5a-0ed9-445f-9f97-337eb2726370-kube-api-access-kxm45\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:18.001446 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.001413    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/419d0a5a-0ed9-445f-9f97-337eb2726370-metrics-certs\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:18.001590 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.001480    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/419d0a5a-0ed9-445f-9f97-337eb2726370-webhook-cert\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:18.006532 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.006506    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxm45\" (UniqueName: \"kubernetes.io/projected/419d0a5a-0ed9-445f-9f97-337eb2726370-kube-api-access-kxm45\") pod \"maas-controller-55b7c7d477-j6zng\" (UID: \"419d0a5a-0ed9-445f-9f97-337eb2726370\") " pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:18.022179 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.022159    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:18.294518 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.294491    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-55b7c7d477-j6zng"]
Sep 10 14:56:18.296853 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:56:18.296814    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419d0a5a_0ed9_445f_9f97_337eb2726370.slice/crio-f24ed130c231055df862fdaaba224eb90d142c2f109c78ea0634fdeb81c20a00 WatchSource:0}: Error finding container f24ed130c231055df862fdaaba224eb90d142c2f109c78ea0634fdeb81c20a00: Status 404 returned error can't find the container with id f24ed130c231055df862fdaaba224eb90d142c2f109c78ea0634fdeb81c20a00
Sep 10 14:56:18.298235 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.298215    2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 10 14:56:18.725796 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.725733    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ce84b34a-6340-47f7-8d08-8912fc82f4ce" path="/var/lib/kubelet/pods/ce84b34a-6340-47f7-8d08-8912fc82f4ce/volumes"
Sep 10 14:56:18.975763 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.975654    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-j6zng" event={"ID":"419d0a5a-0ed9-445f-9f97-337eb2726370","Type":"ContainerStarted","Data":"cd75559e9f8bff1662ad9dc7fdd5ba0f1f4ab24ed8173c0423a723a876b695f6"}
Sep 10 14:56:18.975763 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.975706    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-55b7c7d477-j6zng" event={"ID":"419d0a5a-0ed9-445f-9f97-337eb2726370","Type":"ContainerStarted","Data":"f24ed130c231055df862fdaaba224eb90d142c2f109c78ea0634fdeb81c20a00"}
Sep 10 14:56:18.975982 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.975836    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:56:18.992819 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:18.992767    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-55b7c7d477-j6zng" podStartSLOduration=1.6519705820000001 podStartE2EDuration="1.992734811s" totalImagesPullingTime="340.764229ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:56:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:56:18.990773802 +0000 UTC m=+2184.758496579" watchObservedRunningTime="2026-09-10 14:56:18.992734811 +0000 UTC m=+2184.760457591"
Sep 10 14:56:24.160771 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:24.160705    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:56:29.984015 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:56:29.983977    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-55b7c7d477-j6zng"
Sep 10 14:57:34.175651 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.175615    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"]
Sep 10 14:57:34.176221 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.175838    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx" podUID="f24673a9-6616-43ae-9993-903cb25fe08b" containerName="manager" containerID="cri-o://c00886975f8bd10bc0577469f87b5134b0700b22c81ec25acb398f3b9a6c2498" gracePeriod=10
Sep 10 14:57:34.355188 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.355156    2578 generic.go:356] "Generic (PLEG): container finished" podID="f24673a9-6616-43ae-9993-903cb25fe08b" containerID="c00886975f8bd10bc0577469f87b5134b0700b22c81ec25acb398f3b9a6c2498" exitCode=0
Sep 10 14:57:34.355311 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.355229    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx" event={"ID":"f24673a9-6616-43ae-9993-903cb25fe08b","Type":"ContainerDied","Data":"c00886975f8bd10bc0577469f87b5134b0700b22c81ec25acb398f3b9a6c2498"}
Sep 10 14:57:34.413249 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.413228    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:57:34.449314 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.449254    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24673a9-6616-43ae-9993-903cb25fe08b-extensions-socket-volume\") pod \"f24673a9-6616-43ae-9993-903cb25fe08b\" (UID: \"f24673a9-6616-43ae-9993-903cb25fe08b\") "
Sep 10 14:57:34.449314 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.449290    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v5lbk\" (UniqueName: \"kubernetes.io/projected/f24673a9-6616-43ae-9993-903cb25fe08b-kube-api-access-v5lbk\") pod \"f24673a9-6616-43ae-9993-903cb25fe08b\" (UID: \"f24673a9-6616-43ae-9993-903cb25fe08b\") "
Sep 10 14:57:34.449667 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.449642    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f24673a9-6616-43ae-9993-903cb25fe08b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f24673a9-6616-43ae-9993-903cb25fe08b" (UID: "f24673a9-6616-43ae-9993-903cb25fe08b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 10 14:57:34.451986 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.451963    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f24673a9-6616-43ae-9993-903cb25fe08b-kube-api-access-v5lbk" (OuterVolumeSpecName: "kube-api-access-v5lbk") pod "f24673a9-6616-43ae-9993-903cb25fe08b" (UID: "f24673a9-6616-43ae-9993-903cb25fe08b"). InnerVolumeSpecName "kube-api-access-v5lbk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 14:57:34.549961 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.549933    2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f24673a9-6616-43ae-9993-903cb25fe08b-extensions-socket-volume\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:57:34.549961 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:34.549958    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v5lbk\" (UniqueName: \"kubernetes.io/projected/f24673a9-6616-43ae-9993-903cb25fe08b-kube-api-access-v5lbk\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 14:57:35.361506 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:35.361480    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"
Sep 10 14:57:35.361926 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:35.361474    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx" event={"ID":"f24673a9-6616-43ae-9993-903cb25fe08b","Type":"ContainerDied","Data":"cc211167b013afbe3b6a51767d58d743f1208a337258132c000700e7e8ca67f4"}
Sep 10 14:57:35.361926 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:35.361613    2578 scope.go:117] "RemoveContainer" containerID="c00886975f8bd10bc0577469f87b5134b0700b22c81ec25acb398f3b9a6c2498"
Sep 10 14:57:35.377079 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:35.377055    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"]
Sep 10 14:57:35.380347 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:35.380329    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-bm9qx"]
Sep 10 14:57:36.724254 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:36.724221    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f24673a9-6616-43ae-9993-903cb25fe08b" path="/var/lib/kubelet/pods/f24673a9-6616-43ae-9993-903cb25fe08b/volumes"
Sep 10 14:57:42.485141 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:57:42.485092    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:58:34.820150 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.820112    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"]
Sep 10 14:58:34.820802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.820710    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f24673a9-6616-43ae-9993-903cb25fe08b" containerName="manager"
Sep 10 14:58:34.820802 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.820732    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f24673a9-6616-43ae-9993-903cb25fe08b" containerName="manager"
Sep 10 14:58:34.820936 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.820863    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f24673a9-6616-43ae-9993-903cb25fe08b" containerName="manager"
Sep 10 14:58:34.823973 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.823951    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:34.826499 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.826471    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-8tcdd\""
Sep 10 14:58:34.836867 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.836841    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"]
Sep 10 14:58:34.928001 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.927958    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4cbc986-f015-4dce-9907-4829818e1759-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mcgxf\" (UID: \"e4cbc986-f015-4dce-9907-4829818e1759\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:34.928183 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:34.928031    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgd8d\" (UniqueName: \"kubernetes.io/projected/e4cbc986-f015-4dce-9907-4829818e1759-kube-api-access-lgd8d\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mcgxf\" (UID: \"e4cbc986-f015-4dce-9907-4829818e1759\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:35.028635 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.028596    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgd8d\" (UniqueName: \"kubernetes.io/projected/e4cbc986-f015-4dce-9907-4829818e1759-kube-api-access-lgd8d\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mcgxf\" (UID: \"e4cbc986-f015-4dce-9907-4829818e1759\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:35.028831 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.028677    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4cbc986-f015-4dce-9907-4829818e1759-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mcgxf\" (UID: \"e4cbc986-f015-4dce-9907-4829818e1759\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:35.029034 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.029017    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e4cbc986-f015-4dce-9907-4829818e1759-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mcgxf\" (UID: \"e4cbc986-f015-4dce-9907-4829818e1759\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:35.036707 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.036677    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgd8d\" (UniqueName: \"kubernetes.io/projected/e4cbc986-f015-4dce-9907-4829818e1759-kube-api-access-lgd8d\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mcgxf\" (UID: \"e4cbc986-f015-4dce-9907-4829818e1759\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:35.134816 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.134682    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:35.427572 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.427545    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"]
Sep 10 14:58:35.430365 ip-10-0-142-68 kubenswrapper[2578]: W0910 14:58:35.430337    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4cbc986_f015_4dce_9907_4829818e1759.slice/crio-dd1ab4ca9c06ca0809366ca5019c72eae7273512405b81a27e0b6f195e13ed42 WatchSource:0}: Error finding container dd1ab4ca9c06ca0809366ca5019c72eae7273512405b81a27e0b6f195e13ed42: Status 404 returned error can't find the container with id dd1ab4ca9c06ca0809366ca5019c72eae7273512405b81a27e0b6f195e13ed42
Sep 10 14:58:35.659189 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.659156    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf" event={"ID":"e4cbc986-f015-4dce-9907-4829818e1759","Type":"ContainerStarted","Data":"b8e6111de370b7cbed9f7b608f43548f43488967274ae7b68101a973915324bf"}
Sep 10 14:58:35.659385 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.659207    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf" event={"ID":"e4cbc986-f015-4dce-9907-4829818e1759","Type":"ContainerStarted","Data":"dd1ab4ca9c06ca0809366ca5019c72eae7273512405b81a27e0b6f195e13ed42"}
Sep 10 14:58:35.659385 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.659241    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:58:35.678508 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:35.678410    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf" podStartSLOduration=1.678391373 podStartE2EDuration="1.678391373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 14:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 14:58:35.677143175 +0000 UTC m=+2321.444865950" watchObservedRunningTime="2026-09-10 14:58:35.678391373 +0000 UTC m=+2321.446114155"
Sep 10 14:58:46.668179 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:58:46.668095    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mcgxf"
Sep 10 14:59:27.362515 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:59:27.362473    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:59:33.734455 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:59:33.734414    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:59:50.981440 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:59:50.981397    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 14:59:51.845437 ip-10-0-142-68 kubenswrapper[2578]: I0910 14:59:51.845397    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:17.936289 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:17.936212    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"]
Sep 10 15:00:17.939782 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:17.939760    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:17.953322 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:17.953299    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"]
Sep 10 15:00:18.103016 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.102992    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbdbc715-81d7-4839-a3ac-a0131478aec2-openshift-service-ca\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.103156 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.103041    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbdbc715-81d7-4839-a3ac-a0131478aec2-metrics-certs\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.103156 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.103065    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7l6zh\" (UniqueName: \"kubernetes.io/projected/bbdbc715-81d7-4839-a3ac-a0131478aec2-kube-api-access-7l6zh\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.103156 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.103082    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbdbc715-81d7-4839-a3ac-a0131478aec2-maas-api-tls\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.204269 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.204196    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbdbc715-81d7-4839-a3ac-a0131478aec2-metrics-certs\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.204269 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.204241    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7l6zh\" (UniqueName: \"kubernetes.io/projected/bbdbc715-81d7-4839-a3ac-a0131478aec2-kube-api-access-7l6zh\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.204269 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.204264    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbdbc715-81d7-4839-a3ac-a0131478aec2-maas-api-tls\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.204499 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.204384    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbdbc715-81d7-4839-a3ac-a0131478aec2-openshift-service-ca\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.205036 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.205014    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bbdbc715-81d7-4839-a3ac-a0131478aec2-openshift-service-ca\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.206765 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.206730    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bbdbc715-81d7-4839-a3ac-a0131478aec2-maas-api-tls\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.206875 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.206838    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bbdbc715-81d7-4839-a3ac-a0131478aec2-metrics-certs\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.213775 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.213732    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7l6zh\" (UniqueName: \"kubernetes.io/projected/bbdbc715-81d7-4839-a3ac-a0131478aec2-kube-api-access-7l6zh\") pod \"maas-api-68f46f5b6f-r8jbm\" (UID: \"bbdbc715-81d7-4839-a3ac-a0131478aec2\") " pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.249629 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.249605    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:18.533617 ip-10-0-142-68 kubenswrapper[2578]: W0910 15:00:18.533588    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbdbc715_81d7_4839_a3ac_a0131478aec2.slice/crio-867051c0bc565f60e00e78578cb8b558afa7633df7b7812a7a828c3174757506 WatchSource:0}: Error finding container 867051c0bc565f60e00e78578cb8b558afa7633df7b7812a7a828c3174757506: Status 404 returned error can't find the container with id 867051c0bc565f60e00e78578cb8b558afa7633df7b7812a7a828c3174757506
Sep 10 15:00:18.534469 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:18.534399    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"]
Sep 10 15:00:19.168432 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:19.168394    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm" event={"ID":"bbdbc715-81d7-4839-a3ac-a0131478aec2","Type":"ContainerStarted","Data":"867051c0bc565f60e00e78578cb8b558afa7633df7b7812a7a828c3174757506"}
Sep 10 15:00:20.175850 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:20.175810    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm" event={"ID":"bbdbc715-81d7-4839-a3ac-a0131478aec2","Type":"ContainerStarted","Data":"fba8c801753a41a0ec9195277a446b154061d08e8d956a78a3cdc88bba3a0b2b"}
Sep 10 15:00:20.176204 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:20.175910    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:20.211725 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:20.211675    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm" podStartSLOduration=2.036977802 podStartE2EDuration="3.211660149s" totalImagesPullingTime="1.174682347s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-10 15:00:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:00:20.20966622 +0000 UTC m=+2425.977389001" watchObservedRunningTime="2026-09-10 15:00:20.211660149 +0000 UTC m=+2425.979382929"
Sep 10 15:00:26.186157 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:26.186121    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-68f46f5b6f-r8jbm"
Sep 10 15:00:26.300842 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:26.300809    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"]
Sep 10 15:00:26.301084 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:26.301044    2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl" podUID="e09018de-91a7-4ed3-b108-10af74121521" containerName="maas-api" containerID="cri-o://2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594" gracePeriod=30
Sep 10 15:00:27.548672 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.548650    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 15:00:27.588616 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.588585    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v5t5\" (UniqueName: \"kubernetes.io/projected/e09018de-91a7-4ed3-b108-10af74121521-kube-api-access-9v5t5\") pod \"e09018de-91a7-4ed3-b108-10af74121521\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") "
Sep 10 15:00:27.588769 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.588649    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e09018de-91a7-4ed3-b108-10af74121521-openshift-service-ca\") pod \"e09018de-91a7-4ed3-b108-10af74121521\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") "
Sep 10 15:00:27.588849 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.588793    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-maas-api-tls\") pod \"e09018de-91a7-4ed3-b108-10af74121521\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") "
Sep 10 15:00:27.588927 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.588900    2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs\") pod \"e09018de-91a7-4ed3-b108-10af74121521\" (UID: \"e09018de-91a7-4ed3-b108-10af74121521\") "
Sep 10 15:00:27.589025 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.588999    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e09018de-91a7-4ed3-b108-10af74121521-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e09018de-91a7-4ed3-b108-10af74121521" (UID: "e09018de-91a7-4ed3-b108-10af74121521"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 10 15:00:27.589208 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.589191    2578 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e09018de-91a7-4ed3-b108-10af74121521-openshift-service-ca\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 15:00:27.591030 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.591000    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e09018de-91a7-4ed3-b108-10af74121521-kube-api-access-9v5t5" (OuterVolumeSpecName: "kube-api-access-9v5t5") pod "e09018de-91a7-4ed3-b108-10af74121521" (UID: "e09018de-91a7-4ed3-b108-10af74121521"). InnerVolumeSpecName "kube-api-access-9v5t5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 10 15:00:27.591281 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.591257    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "e09018de-91a7-4ed3-b108-10af74121521" (UID: "e09018de-91a7-4ed3-b108-10af74121521"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:00:27.591346 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.591331    2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e09018de-91a7-4ed3-b108-10af74121521" (UID: "e09018de-91a7-4ed3-b108-10af74121521"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 10 15:00:27.690188 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.690117    2578 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-metrics-certs\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 15:00:27.690188 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.690151    2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v5t5\" (UniqueName: \"kubernetes.io/projected/e09018de-91a7-4ed3-b108-10af74121521-kube-api-access-9v5t5\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 15:00:27.690188 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:27.690161    2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e09018de-91a7-4ed3-b108-10af74121521-maas-api-tls\") on node \"ip-10-0-142-68.ec2.internal\" DevicePath \"\""
Sep 10 15:00:28.217313 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.217279    2578 generic.go:356] "Generic (PLEG): container finished" podID="e09018de-91a7-4ed3-b108-10af74121521" containerID="2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594" exitCode=0
Sep 10 15:00:28.217527 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.217334    2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"
Sep 10 15:00:28.217527 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.217377    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl" event={"ID":"e09018de-91a7-4ed3-b108-10af74121521","Type":"ContainerDied","Data":"2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594"}
Sep 10 15:00:28.217527 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.217447    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl" event={"ID":"e09018de-91a7-4ed3-b108-10af74121521","Type":"ContainerDied","Data":"f608f8f727a4f393afa7da8fe00687e94f2bf7d4ef935ecbbe5a929f41d9bc68"}
Sep 10 15:00:28.217527 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.217476    2578 scope.go:117] "RemoveContainer" containerID="2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594"
Sep 10 15:00:28.227096 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.227066    2578 scope.go:117] "RemoveContainer" containerID="2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594"
Sep 10 15:00:28.227377 ip-10-0-142-68 kubenswrapper[2578]: E0910 15:00:28.227356    2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594\": container with ID starting with 2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594 not found: ID does not exist" containerID="2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594"
Sep 10 15:00:28.227465 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.227383    2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594"} err="failed to get container status \"2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594\": rpc error: code = NotFound desc = could not find container \"2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594\": container with ID starting with 2cf4f7c52398b23ce50f56aea294d0f101631a3c37eb9b7101ee087776606594 not found: ID does not exist"
Sep 10 15:00:28.248080 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.248054    2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"]
Sep 10 15:00:28.269332 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.269309    2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-6dd849cfd5-b6mpl"]
Sep 10 15:00:28.724424 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:28.724388    2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e09018de-91a7-4ed3-b108-10af74121521" path="/var/lib/kubelet/pods/e09018de-91a7-4ed3-b108-10af74121521/volumes"
Sep 10 15:00:45.378715 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:45.378667    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mnjns_a9446da5-a76a-4045-b2f7-b6461c1fb59d/manager/0.log"
Sep 10 15:00:45.502427 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:45.502399    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d2fr4_37d62584-d7e1-4aed-9fcd-37ffbeaa687a/manager/1.log"
Sep 10 15:00:45.639193 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:45.639129    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-55b7c7d477-j6zng_419d0a5a-0ed9-445f-9f97-337eb2726370/manager/0.log"
Sep 10 15:00:45.999078 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:45.999008    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-755bd7f47-vjwjf_7cb64b04-52f6-4612-b545-ab66d06106f2/copy-manifests/0.log"
Sep 10 15:00:46.010514 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:46.010491    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-755bd7f47-vjwjf_7cb64b04-52f6-4612-b545-ab66d06106f2/manager/0.log"
Sep 10 15:00:46.710710 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:46.710684    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-68f46f5b6f-r8jbm_bbdbc715-81d7-4839-a3ac-a0131478aec2/maas-api/0.log"
Sep 10 15:00:47.044232 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:47.044148    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-sxpth_b097fd19-98a9-4953-861b-5ebbe643cee4/postgres/0.log"
Sep 10 15:00:47.933121 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:47.933086    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-j8glg_54ac7e88-70f7-4bfc-a6a7-7c882ca93715/manager/0.log"
Sep 10 15:00:48.052606 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:48.052577    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-skwbf_54745ef5-fbf2-46c8-afd1-407f20e7c1c0/manager/0.log"
Sep 10 15:00:48.168015 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:48.167993    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-zckzk_75256127-4d2c-4928-a207-07717cfb6c86/kuadrant-console-plugin/0.log"
Sep 10 15:00:48.302379 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:48.302310    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-mcgxf_e4cbc986-f015-4dce-9907-4829818e1759/manager/0.log"
Sep 10 15:00:48.605992 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:48.605959    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-mpn9t_b2f856a7-95c5-4e6c-b07f-59acdb3b3b15/manager/0.log"
Sep 10 15:00:49.349192 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:49.349160    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f69648866-xgxnx_b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f/kube-auth-proxy/0.log"
Sep 10 15:00:49.729273 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:49.729134    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vjxxl_c4ce8ee1-18ff-495f-a1d6-20320ed2118d/payload-processing/0.log"
Sep 10 15:00:50.358793 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:50.358769    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-gf4qb_d8453cc8-efd1-44ac-93b6-0a1dea27db7c/main/0.log"
Sep 10 15:00:50.606103 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:50.606062    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-wlzxh_4546e28e-4ef4-425a-aaa8-8288b34e99f4/main/0.log"
Sep 10 15:00:50.836596 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:50.836569    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-s2fg2_184501d0-aeb8-4313-8358-50c8d1ddb772/main/0.log"
Sep 10 15:00:56.780194 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:56.780158    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 15:00:56.964237 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:00:56.964211    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 15:01:00.300481 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:00.300444    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-4lk2v_3779e8a4-b29e-4703-846e-1a6a3b58c4cd/global-pull-secret-syncer/0.log"
Sep 10 15:01:00.537299 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:00.537269    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log"
Sep 10 15:01:00.800611 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:00.800573    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-68.ec2.internal_579903845495bf197e8392d11f220146/haproxy/0.log"
Sep 10 15:01:05.258401 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:05.258375    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-j8glg_54ac7e88-70f7-4bfc-a6a7-7c882ca93715/manager/0.log"
Sep 10 15:01:05.348569 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:05.348537    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-skwbf_54745ef5-fbf2-46c8-afd1-407f20e7c1c0/manager/0.log"
Sep 10 15:01:05.427366 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:05.427339    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-zckzk_75256127-4d2c-4928-a207-07717cfb6c86/kuadrant-console-plugin/0.log"
Sep 10 15:01:05.532105 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:05.532049    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-mcgxf_e4cbc986-f015-4dce-9907-4829818e1759/manager/0.log"
Sep 10 15:01:05.712538 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:05.712508    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-mpn9t_b2f856a7-95c5-4e6c-b07f-59acdb3b3b15/manager/0.log"
Sep 10 15:01:07.312917 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.312881    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_958214c3-8d97-4848-a91f-a062d0bcbdbb/alertmanager/0.log"
Sep 10 15:01:07.371579 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.371552    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_958214c3-8d97-4848-a91f-a062d0bcbdbb/config-reloader/0.log"
Sep 10 15:01:07.412050 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.412014    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_958214c3-8d97-4848-a91f-a062d0bcbdbb/kube-rbac-proxy-web/0.log"
Sep 10 15:01:07.493035 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.492997    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_958214c3-8d97-4848-a91f-a062d0bcbdbb/kube-rbac-proxy/0.log"
Sep 10 15:01:07.587427 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.587403    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_958214c3-8d97-4848-a91f-a062d0bcbdbb/kube-rbac-proxy-metric/0.log"
Sep 10 15:01:07.629304 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.629275    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_958214c3-8d97-4848-a91f-a062d0bcbdbb/prom-label-proxy/0.log"
Sep 10 15:01:07.666491 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.666467    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_958214c3-8d97-4848-a91f-a062d0bcbdbb/init-config-reloader/0.log"
Sep 10 15:01:07.790241 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.790214    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-v4mlz_3ca48f0c-9fd1-4763-9f96-92cbdd07510e/kube-state-metrics/0.log"
Sep 10 15:01:07.835651 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.835624    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-v4mlz_3ca48f0c-9fd1-4763-9f96-92cbdd07510e/kube-rbac-proxy-main/0.log"
Sep 10 15:01:07.870412 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.870348    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-74cb57df7-v4mlz_3ca48f0c-9fd1-4763-9f96-92cbdd07510e/kube-rbac-proxy-self/0.log"
Sep 10 15:01:07.909600 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:07.909572    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6875cf7786-5fvhn_e248996f-5cc7-4ede-9962-849e03af7eee/metrics-server/0.log"
Sep 10 15:01:08.204323 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:08.204239    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pcxg8_ee6f2e2e-5eab-468d-8bbf-de67d03f1152/node-exporter/0.log"
Sep 10 15:01:08.264941 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:08.264918    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pcxg8_ee6f2e2e-5eab-468d-8bbf-de67d03f1152/kube-rbac-proxy/0.log"
Sep 10 15:01:08.320399 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:08.320374    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-pcxg8_ee6f2e2e-5eab-468d-8bbf-de67d03f1152/init-textfile/0.log"
Sep 10 15:01:10.227733 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.227698    2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"]
Sep 10 15:01:10.228106 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.228098    2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e09018de-91a7-4ed3-b108-10af74121521" containerName="maas-api"
Sep 10 15:01:10.228147 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.228110    2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e09018de-91a7-4ed3-b108-10af74121521" containerName="maas-api"
Sep 10 15:01:10.228186 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.228178    2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e09018de-91a7-4ed3-b108-10af74121521" containerName="maas-api"
Sep 10 15:01:10.231222 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.231197    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.234451 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.234432    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b6dn7\"/\"openshift-service-ca.crt\""
Sep 10 15:01:10.234708 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.234690    2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-b6dn7\"/\"kube-root-ca.crt\""
Sep 10 15:01:10.234848 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.234728    2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-b6dn7\"/\"default-dockercfg-86sqw\""
Sep 10 15:01:10.240545 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.240523    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"]
Sep 10 15:01:10.319638 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.319606    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-proc\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.319638 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.319643    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-lib-modules\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.319863 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.319666    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-podres\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.319863 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.319820    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-sys\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.319863 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.319846    2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9w9kn\" (UniqueName: \"kubernetes.io/projected/60eafba2-9db3-4b2d-ae70-deb8315032cc-kube-api-access-9w9kn\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420254 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420220    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-sys\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420254 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420258    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9w9kn\" (UniqueName: \"kubernetes.io/projected/60eafba2-9db3-4b2d-ae70-deb8315032cc-kube-api-access-9w9kn\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420454 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420282    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-proc\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420454 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-proc\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420454 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420349    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-sys\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420454 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420429    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-lib-modules\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420625 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420469    2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-podres\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420625 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420577    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-lib-modules\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.420625 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.420612    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/60eafba2-9db3-4b2d-ae70-deb8315032cc-podres\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.430648 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.430618    2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9w9kn\" (UniqueName: \"kubernetes.io/projected/60eafba2-9db3-4b2d-ae70-deb8315032cc-kube-api-access-9w9kn\") pod \"perf-node-gather-daemonset-hgxqh\" (UID: \"60eafba2-9db3-4b2d-ae70-deb8315032cc\") " pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.541306 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.541223    2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:10.830408 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:10.830346    2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"]
Sep 10 15:01:10.833242 ip-10-0-142-68 kubenswrapper[2578]: W0910 15:01:10.833213    2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod60eafba2_9db3_4b2d_ae70_deb8315032cc.slice/crio-65e8c92573801aacce82e3fd37f5fdfb0544bb697ab59f180c6722dc799ca681 WatchSource:0}: Error finding container 65e8c92573801aacce82e3fd37f5fdfb0544bb697ab59f180c6722dc799ca681: Status 404 returned error can't find the container with id 65e8c92573801aacce82e3fd37f5fdfb0544bb697ab59f180c6722dc799ca681
Sep 10 15:01:11.430149 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:11.430110    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh" event={"ID":"60eafba2-9db3-4b2d-ae70-deb8315032cc","Type":"ContainerStarted","Data":"e1f502a17a14c60af07178aa66cd66011900822dd6834039f8dcb934f4a28f08"}
Sep 10 15:01:11.430541 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:11.430221    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:11.430541 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:11.430246    2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh" event={"ID":"60eafba2-9db3-4b2d-ae70-deb8315032cc","Type":"ContainerStarted","Data":"65e8c92573801aacce82e3fd37f5fdfb0544bb697ab59f180c6722dc799ca681"}
Sep 10 15:01:11.451331 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:11.451280    2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh" podStartSLOduration=1.451264014 podStartE2EDuration="1.451264014s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-10 15:01:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-10 15:01:11.45002261 +0000 UTC m=+2477.217745390" watchObservedRunningTime="2026-09-10 15:01:11.451264014 +0000 UTC m=+2477.218986793"
Sep 10 15:01:13.356122 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:13.356095    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kw77b_a78995de-d2c7-4b46-9118-e1187aba51e9/dns/0.log"
Sep 10 15:01:13.379382 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:13.379357    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-kw77b_a78995de-d2c7-4b46-9118-e1187aba51e9/kube-rbac-proxy/0.log"
Sep 10 15:01:13.486215 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:13.486195    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kk5vd_607b65d4-332c-465a-92af-3363fc98146b/dns-node-resolver/0.log"
Sep 10 15:01:14.072256 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:14.072221    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-8657cbd865-5nzzg_ffb246e9-d58f-439c-904b-07a3ce72d2ca/registry/0.log"
Sep 10 15:01:14.164865 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:14.164835    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-cdx6z_ad4815b3-1e11-427b-86f1-f65145c8cf90/node-ca/0.log"
Sep 10 15:01:15.458573 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:15.458546    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f69648866-xgxnx_b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f/kube-auth-proxy/0.log"
Sep 10 15:01:15.584297 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:15.584271    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vjxxl_c4ce8ee1-18ff-495f-a1d6-20320ed2118d/payload-processing/0.log"
Sep 10 15:01:16.579385 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:16.579361    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ptbns_91b7b77a-784b-4ba5-9880-c54262d9c27c/serve-healthcheck-canary/0.log"
Sep 10 15:01:17.188929 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:17.188899    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-65f2t_96655b5e-c11e-46fa-b7f8-c68350542eaa/kube-rbac-proxy/0.log"
Sep 10 15:01:17.217159 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:17.217118    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-65f2t_96655b5e-c11e-46fa-b7f8-c68350542eaa/exporter/0.log"
Sep 10 15:01:17.243793 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:17.243765    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-65f2t_96655b5e-c11e-46fa-b7f8-c68350542eaa/extractor/0.log"
Sep 10 15:01:17.445160 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:17.445106    2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-b6dn7/perf-node-gather-daemonset-hgxqh"
Sep 10 15:01:19.825239 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:19.825203    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-mnjns_a9446da5-a76a-4045-b2f7-b6461c1fb59d/manager/0.log"
Sep 10 15:01:19.911787 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:19.911735    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d2fr4_37d62584-d7e1-4aed-9fcd-37ffbeaa687a/manager/1.log"
Sep 10 15:01:20.122404 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:20.122334    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-d2fr4_37d62584-d7e1-4aed-9fcd-37ffbeaa687a/manager/0.log"
Sep 10 15:01:20.194570 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:20.194536    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-55b7c7d477-j6zng_419d0a5a-0ed9-445f-9f97-337eb2726370/manager/0.log"
Sep 10 15:01:20.390786 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:20.390693    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-755bd7f47-vjwjf_7cb64b04-52f6-4612-b545-ab66d06106f2/manager/0.log"
Sep 10 15:01:20.415838 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:20.415821    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-755bd7f47-vjwjf_7cb64b04-52f6-4612-b545-ab66d06106f2/copy-manifests/0.log"
Sep 10 15:01:20.869285 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:20.869261    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f69648866-xgxnx_b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f/kube-auth-proxy/0.log"
Sep 10 15:01:20.956474 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:20.956442    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vjxxl_c4ce8ee1-18ff-495f-a1d6-20320ed2118d/payload-processing/0.log"
Sep 10 15:01:21.473670 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:21.473635    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 15:01:21.977731 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:21.977485    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-tbcnp_d1bc2c98-b7e8-4b15-a39c-d532dcb85455/openshift-lws-operator/0.log"
Sep 10 15:01:22.762293 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:22.762257    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6f69648866-xgxnx_b3c5e213-92b0-4ef3-a8b7-15fa0c5cca6f/kube-auth-proxy/0.log"
Sep 10 15:01:22.859982 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:22.859951    2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-vjxxl_c4ce8ee1-18ff-495f-a1d6-20320ed2118d/payload-processing/0.log"
Sep 10 15:01:25.304710 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:25.304660    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024
Sep 10 15:01:25.379441 ip-10-0-142-68 kubenswrapper[2578]: I0910 15:01:25.379410    2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-5h8xw_717ed4b3-2706-4ed4-a2cb-dae3040d7e30/konnectivity-agent/0.log" limit=1024