Jul 28 19:33:00.550144 ip-10-0-128-255 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 28 19:33:00.550154 ip-10-0-128-255 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 28 19:33:00.550162 ip-10-0-128-255 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 28 19:33:00.550399 ip-10-0-128-255 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 28 19:33:10.664671 ip-10-0-128-255 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 28 19:33:10.664699 ip-10-0-128-255 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 01864c1103ad47e39d459ec241118ab7 --
Jul 28 19:34:45.490714 ip-10-0-128-255 systemd[1]: Starting Kubernetes Kubelet...
Jul 28 19:34:45.831407 ip-10-0-128-255 kubenswrapper[2568]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:34:45.831407 ip-10-0-128-255 kubenswrapper[2568]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 28 19:34:45.831407 ip-10-0-128-255 kubenswrapper[2568]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:34:45.831407 ip-10-0-128-255 kubenswrapper[2568]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 28 19:34:45.831407 ip-10-0-128-255 kubenswrapper[2568]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 28 19:34:45.833288 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.833197    2568 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 28 19:34:45.837479 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837462    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:34:45.837479 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837479    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837483    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837487    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837490    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837492    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837496    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837499    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837503    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837511    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837514    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837517    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837520    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837522    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837525    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837527    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837530    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837533    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837535    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837538    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837540    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:34:45.837540 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837543    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837546    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837549    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837552    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837555    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837558    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837560    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837563    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837565    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837568    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837570    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837573    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837578    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837582    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837585    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837587    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837590    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837593    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837595    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:34:45.838017 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837598    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837600    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837603    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837606    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837609    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837611    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837613    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837616    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837619    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837621    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837624    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837626    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837629    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837631    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837634    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837637    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837640    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837642    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837645    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837647    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:34:45.838491 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837650    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837652    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837655    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837657    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837660    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837662    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837665    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837668    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837670    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837672    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837675    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837677    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837680    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837683    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837687    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837691    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837694    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837697    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837700    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:34:45.838971 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837702    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837705    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837707    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837710    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837712    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837715    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837717    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837719    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837723    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837726    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837728    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837731    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837733    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837736    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837740    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837744    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837746    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837749    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837752    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.837754    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:34:45.839697 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838542    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838549    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838552    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838554    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838557    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838559    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838563    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838565    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838568    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838577    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838579    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838582    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838585    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838587    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838590    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838592    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838595    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838597    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838600    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:34:45.840299 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838602    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838607    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838610    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838612    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838615    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838618    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838620    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838623    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838626    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838628    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838631    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838633    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838636    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838638    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838641    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838643    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838646    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838648    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838651    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838654    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:34:45.840814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838656    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838658    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838661    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838669    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838671    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838674    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838678    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838682    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838685    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838689    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838692    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838695    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838697    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838701    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838705    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838707    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838710    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838712    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838715    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:34:45.841313 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838717    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838720    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838722    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838725    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838727    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838729    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838732    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838735    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838737    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838740    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838742    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838745    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838747    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838751    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838754    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838756    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838759    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838767    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838769    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838772    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:34:45.841814 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838774    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838777    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838779    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838782    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838785    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838787    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838790    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838793    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838796    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838798    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838801    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838805    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838825    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838829    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838833    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838836    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838841    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838844    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838847    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838850    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:34:45.842325 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.838853    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840835    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840846    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840850    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840853    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840856    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840858    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840861    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840863    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840866    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840869    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840872    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840884    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840887    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840890    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840892    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840896    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840899    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840901    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840904    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:34:45.842830 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840906    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840909    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840911    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840914    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840916    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840919    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840923    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840926    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840929    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840932    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840935    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840937    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840943    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840946    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840949    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840952    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840954    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840957    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840959    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:34:45.843317 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840962    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840964    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840967    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840970    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840972    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840975    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840977    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840980    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840982    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840985    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840987    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840990    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840993    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840996    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.840998    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841001    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841004    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841006    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841009    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841011    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:34:45.843825 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841014    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841017    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841019    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841022    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841025    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841028    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841031    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841034    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841037    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841040    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841042    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841045    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841048    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841050    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841053    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841055    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841059    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841063    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841066    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841069    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:34:45.844309 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841072    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841075    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841078    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841081    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841083    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841087    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841090    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841093    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841095    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841098    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841100    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841103    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841105    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841108    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841110    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841113    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841116    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841118    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841122    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:34:45.844805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841126    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841129    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841202    2568 flags.go:64] FLAG: --address="0.0.0.0"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841209    2568 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841216    2568 flags.go:64] FLAG: --anonymous-auth="true"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841220    2568 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841233    2568 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841236    2568 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841241    2568 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841245    2568 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841248    2568 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841251    2568 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841254    2568 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841258    2568 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841261    2568 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841264    2568 flags.go:64] FLAG: --cgroup-root=""
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841267    2568 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841270    2568 flags.go:64] FLAG: --client-ca-file=""
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841272    2568 flags.go:64] FLAG: --cloud-provider="external"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841276    2568 flags.go:64] FLAG: --cluster-dns="[]"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841280    2568 flags.go:64] FLAG: --cluster-domain=""
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841283    2568 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841287    2568 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841290    2568 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 28 19:34:45.845298 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841293    2568 flags.go:64] FLAG: --container-log-max-files="5"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841301    2568 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841304    2568 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841308    2568 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841311    2568 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841315    2568 flags.go:64] FLAG: --contention-profiling="false"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841318    2568 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841321    2568 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841325    2568 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841328    2568 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841333    2568 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841336    2568 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841339    2568 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841342    2568 flags.go:64] FLAG: --enable-load-reader="false"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841345    2568 flags.go:64] FLAG: --enable-server="true"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841347    2568 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841352    2568 flags.go:64] FLAG: --event-burst="100"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841355    2568 flags.go:64] FLAG: --event-qps="50"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841358    2568 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841361    2568 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841364    2568 flags.go:64] FLAG: --eviction-hard=""
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841367    2568 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841370    2568 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841373    2568 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841383    2568 flags.go:64] FLAG: --eviction-soft=""
Jul 28 19:34:45.845886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841385    2568 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841388    2568 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841391    2568 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841395    2568 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841397    2568 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841400    2568 flags.go:64] FLAG: --fail-swap-on="true"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841403    2568 flags.go:64] FLAG: --feature-gates=""
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841407    2568 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841409    2568 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841413    2568 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841430    2568 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841435    2568 flags.go:64] FLAG: --healthz-port="10248"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841438    2568 flags.go:64] FLAG: --help="false"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841441    2568 flags.go:64] FLAG: --hostname-override="ip-10-0-128-255.ec2.internal"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841444    2568 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841447    2568 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841451    2568 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841454    2568 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841458    2568 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841461    2568 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841464    2568 flags.go:64] FLAG: --image-service-endpoint=""
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841467    2568 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841469    2568 flags.go:64] FLAG: --kube-api-burst="100"
Jul 28 19:34:45.846547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841472    2568 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841475    2568 flags.go:64] FLAG: --kube-api-qps="50"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841478    2568 flags.go:64] FLAG: --kube-reserved=""
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841481    2568 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841484    2568 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841487    2568 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841489    2568 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841492    2568 flags.go:64] FLAG: --lock-file=""
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841495    2568 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841498    2568 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841501    2568 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841506    2568 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841509    2568 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841513    2568 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841515    2568 flags.go:64] FLAG: --logging-format="text"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841518    2568 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841522    2568 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841529    2568 flags.go:64] FLAG: --manifest-url=""
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841532    2568 flags.go:64] FLAG: --manifest-url-header=""
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841537    2568 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841540    2568 flags.go:64] FLAG: --max-open-files="1000000"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841544    2568 flags.go:64] FLAG: --max-pods="110"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841548    2568 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841551    2568 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841554    2568 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 28 19:34:45.847119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841557    2568 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841560    2568 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841563    2568 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841566    2568 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841573    2568 flags.go:64] FLAG: --node-status-max-images="50"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841576    2568 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841579    2568 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841582    2568 flags.go:64] FLAG: --pod-cidr=""
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841585    2568 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841590    2568 flags.go:64] FLAG: --pod-manifest-path=""
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841593    2568 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841596    2568 flags.go:64] FLAG: --pods-per-core="0"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841599    2568 flags.go:64] FLAG: --port="10250"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841602    2568 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841605    2568 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0ca09261d128f699d"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841608    2568 flags.go:64] FLAG: --qos-reserved=""
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841611    2568 flags.go:64] FLAG: --read-only-port="10255"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841614    2568 flags.go:64] FLAG: --register-node="true"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841617    2568 flags.go:64] FLAG: --register-with-taints=""
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841621    2568 flags.go:64] FLAG: --registry-burst="10"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841624    2568 flags.go:64] FLAG: --registry-qps="5"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841627    2568 flags.go:64] FLAG: --reserved-cpus=""
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841630    2568 flags.go:64] FLAG: --reserved-memory=""
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841633    2568 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841637    2568 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 28 19:34:45.847739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841640    2568 flags.go:64] FLAG: --rotate-certificates="false"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841643    2568 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841646    2568 flags.go:64] FLAG: --runonce="false"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841649    2568 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841655    2568 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841658    2568 flags.go:64] FLAG: --seccomp-default="false"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841660    2568 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841663    2568 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841667    2568 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841670    2568 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841673    2568 flags.go:64] FLAG: --storage-driver-password="root"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841676    2568 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841679    2568 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841682    2568 flags.go:64] FLAG: --storage-driver-user="root"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841685    2568 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841688    2568 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841691    2568 flags.go:64] FLAG: --system-cgroups=""
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841694    2568 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841699    2568 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841702    2568 flags.go:64] FLAG: --tls-cert-file=""
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841705    2568 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841732    2568 flags.go:64] FLAG: --tls-min-version=""
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841736    2568 flags.go:64] FLAG: --tls-private-key-file=""
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841740    2568 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841743    2568 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 28 19:34:45.848348 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841746    2568 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841749    2568 flags.go:64] FLAG: --v="2"
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841753    2568 flags.go:64] FLAG: --version="false"
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841757    2568 flags.go:64] FLAG: --vmodule=""
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841762    2568 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.841766    2568 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841888    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841893    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841896    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841899    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841902    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841906    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841908    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841911    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841917    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841920    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841924    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841928    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841931    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841934    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841937    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:34:45.849015 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841939    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841942    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841945    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841947    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841950    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841953    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841955    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841958    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841961    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841963    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841966    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841968    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841971    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841973    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841976    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841978    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841982    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841984    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841987    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841990    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:34:45.849547 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841992    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841994    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.841998    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842001    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842003    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842007    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842010    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842012    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842015    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842018    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842020    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842023    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842025    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842028    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842030    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842033    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842036    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842038    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842040    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842043    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:34:45.850029 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842046    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842048    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842051    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842053    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842055    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842058    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842060    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842063    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842065    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842069    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842072    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842074    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842077    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842079    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842083    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842085    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842088    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842092    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842096    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842099    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:34:45.850533 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842102    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842105    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842108    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842111    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842113    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842116    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842119    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842123    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842126    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842128    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842131    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842134    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842136    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842139    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842141    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842144    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842146    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842149    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842151    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:34:45.851014 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842154    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842156    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842159    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842162    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.842165    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.843441    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.850453    2568 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.850469    2568 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.850488    2568 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.850498    2568 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850553    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850559    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850563    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850566    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850570    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:34:45.851559 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850573    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850577    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850580    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850582    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850585    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850588    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850591    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850593    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850596    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850599    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850601    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850604    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850606    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850610    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850614    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850617    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850620    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850623    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850625    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:34:45.851937 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850628    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850632    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850636    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850639    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850642    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850646    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850649    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850652    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850654    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850657    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850660    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850662    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850665    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850668    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850671    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850673    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850676    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850679    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850682    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850684    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:34:45.852395 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850687    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850689    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850692    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850695    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850697    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850700    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850702    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850705    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850708    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850711    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850713    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850715    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850718    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850720    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850723    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850726    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850729    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850731    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850734    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850737    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:34:45.852891 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850739    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850742    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850744    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850748    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850751    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850754    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850757    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850760    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850763    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850765    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850768    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850771    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850773    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850776    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850778    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850781    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850784    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850786    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850788    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:34:45.853472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850791    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850794    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850796    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850799    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850802    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850804    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850807    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850809    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850816    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850819    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850822    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850825    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850828    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850830    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850833    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:34:45.853932 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850836    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.850841    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850967    2568 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850971    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850974    2568 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850977    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850980    2568 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850984    2568 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850988    2568 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850991    2568 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850994    2568 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850996    2568 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.850999    2568 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851002    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851004    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 28 19:34:45.854305 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851007    2568 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851010    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851012    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851015    2568 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851018    2568 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851020    2568 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851023    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851026    2568 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851028    2568 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851031    2568 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851034    2568 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851037    2568 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851040    2568 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851043    2568 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851045    2568 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851048    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851051    2568 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851053    2568 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851056    2568 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851058    2568 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 28 19:34:45.854787 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851061    2568 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851063    2568 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851066    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851068    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851071    2568 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851073    2568 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851076    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851078    2568 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851081    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851083    2568 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851086    2568 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851088    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851092    2568 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851095    2568 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851098    2568 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851101    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851104    2568 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851107    2568 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851110    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 28 19:34:45.855266 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851112    2568 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851115    2568 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851118    2568 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851121    2568 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851124    2568 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851127    2568 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851130    2568 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851132    2568 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851135    2568 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851137    2568 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851140    2568 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851143    2568 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851145    2568 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851148    2568 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851150    2568 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851153    2568 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851155    2568 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851159    2568 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851162    2568 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851165    2568 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 28 19:34:45.855750 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851167    2568 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851170    2568 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851172    2568 feature_gate.go:328] unrecognized feature gate: Example
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851175    2568 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851177    2568 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851180    2568 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851182    2568 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851185    2568 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851188    2568 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851190    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851192    2568 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851195    2568 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851197    2568 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851200    2568 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851205    2568 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851207    2568 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851210    2568 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851213    2568 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851216    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851219    2568 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 28 19:34:45.856228 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851221    2568 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851224    2568 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851227    2568 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851229    2568 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851232    2568 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851234    2568 feature_gate.go:328] unrecognized feature gate: Example2
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:45.851237    2568 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.851241    2568 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.851344    2568 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.853788    2568 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.854978    2568 server.go:1019] "Starting client certificate rotation"
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.855081    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:34:45.856727 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.855116    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 28 19:34:45.857963 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.857947    2568 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:34:45.859269 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.859256    2568 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 28 19:34:45.867385 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.867358    2568 log.go:25] "Validated CRI v1 runtime API"
Jul 28 19:34:45.870644 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.870627    2568 log.go:25] "Validated CRI v1 image API"
Jul 28 19:34:45.873654 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.873635    2568 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 28 19:34:45.875891 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.875870    2568 fs.go:135] Filesystem UUIDs: map[273186ec-159f-42d4-91a1-4af3700c0700:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7dd65750-564e-4f8e-a21b-a3c2a7beb2ce:/dev/nvme0n1p3]
Jul 28 19:34:45.875963 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.875890    2568 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 28 19:34:45.881718 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.881607    2568 manager.go:216] Machine: {Timestamp:2026-07-28 19:34:45.880350036 +0000 UTC m=+0.286814523 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3087500 MemoryCapacity:33164492800 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2bed05fcf45cc55c03679935cd50bd SystemUUID:ec2bed05-fcf4-5cc5-5c03-679935cd50bd BootID:01864c11-03ad-47e3-9d45-9ec241118ab7 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e2:c0:48:fc:29 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e2:c0:48:fc:29 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6a:27:34:21:60:ce Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164492800 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 28 19:34:45.881718 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.881713    2568 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 28 19:34:45.881840 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.881799    2568 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 28 19:34:45.884365 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.884332    2568 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 28 19:34:45.884523 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.884367    2568 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-255.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 28 19:34:45.884646 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.884631    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:34:45.885099 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.885088    2568 topology_manager.go:138] "Creating topology manager with none policy"
Jul 28 19:34:45.885134 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.885103    2568 container_manager_linux.go:309] "Creating device plugin manager"
Jul 28 19:34:45.885134 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.885111    2568 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:34:45.885134 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.885125    2568 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 28 19:34:45.886410 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.886400    2568 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 28 19:34:45.891069 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891058    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:34:45.891193 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891184    2568 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 28 19:34:45.891225 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891215    2568 kubelet.go:486] "Attempting to sync node with API server"
Jul 28 19:34:45.891253 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891229    2568 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 28 19:34:45.891286 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891253    2568 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 28 19:34:45.891286 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891267    2568 kubelet.go:398] "Adding apiserver pod source"
Jul 28 19:34:45.891286 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891280    2568 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 28 19:34:45.891711 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891699    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:34:45.891787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.891717    2568 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 28 19:34:45.893615 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.893590    2568 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 28 19:34:45.895137 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895124    2568 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 28 19:34:45.895195 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895154    2568 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 28 19:34:45.895753 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895740    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 28 19:34:45.895794 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895763    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 28 19:34:45.895794 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895773    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 28 19:34:45.895794 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895780    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 28 19:34:45.895794 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895790    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 28 19:34:45.895794 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895796    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 28 19:34:45.895943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895804    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 28 19:34:45.895943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895810    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 28 19:34:45.895943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895817    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 28 19:34:45.895943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895823    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 28 19:34:45.895943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895832    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 28 19:34:45.895943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.895837    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 28 19:34:45.896468 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.896458    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 28 19:34:45.896468 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.896468    2568 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 28 19:34:45.899539 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.899525    2568 server.go:1268] "Started kubelet"
Jul 28 19:34:45.900013 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.899964    2568 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 28 19:34:45.900485 ip-10-0-128-255 systemd[1]: Started Kubernetes Kubelet.
Jul 28 19:34:45.900707 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.900651    2568 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 28 19:34:45.900919 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.900729    2568 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 28 19:34:45.904025 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.904007    2568 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 28 19:34:45.904394 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.904369    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 28 19:34:45.904568 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.904542    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-255.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 28 19:34:45.904646 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.904630    2568 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-255.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 28 19:34:45.904731 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.904713    2568 server.go:310] "Adding debug handlers to kubelet server"
Jul 28 19:34:45.905576 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.905553    2568 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 28 19:34:45.907805 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.906893    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-255.ec2.internal.18c68b23452a561a  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-255.ec2.internal,UID:ip-10-0-128-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-255.ec2.internal,},FirstTimestamp:2026-07-28 19:34:45.899499034 +0000 UTC m=+0.305963501,LastTimestamp:2026-07-28 19:34:45.899499034 +0000 UTC m=+0.305963501,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-255.ec2.internal,}"
Jul 28 19:34:45.912247 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.912228    2568 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 28 19:34:45.912247 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.912241    2568 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 28 19:34:45.912388 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.912321    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:45.913006 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.912990    2568 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 28 19:34:45.913006 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.912994    2568 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 28 19:34:45.913006 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913005    2568 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 28 19:34:45.913181 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913109    2568 reconstruct.go:97] "Volume reconstruction finished"
Jul 28 19:34:45.913181 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913122    2568 reconciler.go:29] "Reconciler: start to sync state"
Jul 28 19:34:45.913181 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913109    2568 factory.go:55] Registering systemd factory
Jul 28 19:34:45.913181 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913144    2568 factory.go:223] Registration of the systemd container factory successfully
Jul 28 19:34:45.913431 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913398    2568 factory.go:153] Registering CRI-O factory
Jul 28 19:34:45.913431 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913431    2568 factory.go:223] Registration of the crio container factory successfully
Jul 28 19:34:45.913566 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913486    2568 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 28 19:34:45.913566 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913509    2568 factory.go:103] Registering Raw factory
Jul 28 19:34:45.913566 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913524    2568 manager.go:1196] Started watching for new ooms in manager
Jul 28 19:34:45.913925 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.913911    2568 manager.go:349] Starting recovery of all containers
Jul 28 19:34:45.914278 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.914258    2568 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Jul 28 19:34:45.914357 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.914339    2568 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-255.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Jul 28 19:34:45.924190 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.924171    2568 manager.go:354] Recovery completed
Jul 28 19:34:45.928658 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.928643    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:34:45.930990 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.930974    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:34:45.931055 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.931003    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:34:45.931055 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.931012    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:34:45.931579 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.931561    2568 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 28 19:34:45.931579 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.931580    2568 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 28 19:34:45.931682 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.931599    2568 state_mem.go:36] "Initialized new in-memory state store"
Jul 28 19:34:45.931682 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.931572    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:34:45.932474 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.932458    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:34:45.932543 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.932488    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:34:45.932543 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.932502    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:34:45.933918 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.933903    2568 policy_none.go:49] "None policy: Start"
Jul 28 19:34:45.933918 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.933920    2568 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 28 19:34:45.934049 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.933931    2568 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 28 19:34:45.935986 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.935973    2568 policy_none.go:47] "Start"
Jul 28 19:34:45.939816 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.939795    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fgchx"
Jul 28 19:34:45.941690 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.941356    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-255.ec2.internal.18c68b23470ad3e3  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-255.ec2.internal,UID:ip-10-0-128-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-255.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-255.ec2.internal,},FirstTimestamp:2026-07-28 19:34:45.930988515 +0000 UTC m=+0.337452982,LastTimestamp:2026-07-28 19:34:45.930988515 +0000 UTC m=+0.337452982,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-255.ec2.internal,}"
Jul 28 19:34:45.944763 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.944702    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-255.ec2.internal.18c68b23470b1e58  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-255.ec2.internal,UID:ip-10-0-128-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-128-255.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-128-255.ec2.internal,},FirstTimestamp:2026-07-28 19:34:45.931007576 +0000 UTC m=+0.337472042,LastTimestamp:2026-07-28 19:34:45.931007576 +0000 UTC m=+0.337472042,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-255.ec2.internal,}"
Jul 28 19:34:45.946324 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.946304    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fgchx"
Jul 28 19:34:45.951929 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.951839    2568 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-255.ec2.internal.18c68b23470b413f  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-255.ec2.internal,UID:ip-10-0-128-255.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-128-255.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-128-255.ec2.internal,},FirstTimestamp:2026-07-28 19:34:45.931016511 +0000 UTC m=+0.337480977,LastTimestamp:2026-07-28 19:34:45.931016511 +0000 UTC m=+0.337480977,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-255.ec2.internal,}"
Jul 28 19:34:45.973992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.973976    2568 manager.go:337] "Starting Device Plugin manager"
Jul 28 19:34:45.974109 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.974017    2568 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 28 19:34:45.974109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.974029    2568 server.go:87] "Starting device plugin registration server"
Jul 28 19:34:45.974317 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.974304    2568 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 28 19:34:45.974381 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.974322    2568 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 28 19:34:45.974467 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.974450    2568 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 28 19:34:45.974542 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.974531    2568 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 28 19:34:45.974542 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:45.974541    2568 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 28 19:34:45.976700 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.976678    2568 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 28 19:34:45.976809 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:45.976725    2568 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.073447 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.073399    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 28 19:34:46.074644 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.074621    2568 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 28 19:34:46.074644 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.074625    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:34:46.074814 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.074671    2568 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 28 19:34:46.074814 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.074701    2568 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 28 19:34:46.074814 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.074743    2568 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 28 19:34:46.075630 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.075610    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:34:46.075750 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.075649    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:34:46.075750 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.075666    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:34:46.075750 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.075700    2568 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.077344 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.077326    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:34:46.084464 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.084406    2568 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.084532 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.084487    2568 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-255.ec2.internal\": node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.128132 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.127959    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.175275 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.175242    2568 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal"]
Jul 28 19:34:46.175441 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.175330    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:34:46.176384 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.176357    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:34:46.176496 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.176390    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:34:46.176496 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.176403    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:34:46.178884 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.178868    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:34:46.178988 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.178974    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.179037 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.179004    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:34:46.179678 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.179660    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:34:46.179773 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.179690    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:34:46.179773 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.179703    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:34:46.179773 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.179766    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:34:46.179920 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.179785    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:34:46.179920 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.179795    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:34:46.182021 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.182006    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.182101 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.182034    2568 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 28 19:34:46.182819 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.182805    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientMemory"
Jul 28 19:34:46.182888 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.182824    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasNoDiskPressure"
Jul 28 19:34:46.182888 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.182833    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeHasSufficientPID"
Jul 28 19:34:46.213192 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.213168    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-255.ec2.internal\" not found" node="ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.215193 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.215177    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1daed9178f1adc845a4a940fe1f2f13f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal\" (UID: \"1daed9178f1adc845a4a940fe1f2f13f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.215270 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.215201    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1daed9178f1adc845a4a940fe1f2f13f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal\" (UID: \"1daed9178f1adc845a4a940fe1f2f13f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.215270 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.215218    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c45489797f52552f43a8393e08ba8c3a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-255.ec2.internal\" (UID: \"c45489797f52552f43a8393e08ba8c3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.217767 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.217753    2568 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-255.ec2.internal\" not found" node="ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.228875 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.228858    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.316004 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.315967    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1daed9178f1adc845a4a940fe1f2f13f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal\" (UID: \"1daed9178f1adc845a4a940fe1f2f13f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.316104 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.316008    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1daed9178f1adc845a4a940fe1f2f13f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal\" (UID: \"1daed9178f1adc845a4a940fe1f2f13f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.316104 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.316025    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c45489797f52552f43a8393e08ba8c3a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-255.ec2.internal\" (UID: \"c45489797f52552f43a8393e08ba8c3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.316104 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.316068    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c45489797f52552f43a8393e08ba8c3a-config\") pod \"kube-apiserver-proxy-ip-10-0-128-255.ec2.internal\" (UID: \"c45489797f52552f43a8393e08ba8c3a\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.316104 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.316071    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1daed9178f1adc845a4a940fe1f2f13f-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal\" (UID: \"1daed9178f1adc845a4a940fe1f2f13f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.316226 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.316070    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/1daed9178f1adc845a4a940fe1f2f13f-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal\" (UID: \"1daed9178f1adc845a4a940fe1f2f13f\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.329089 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.329064    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.430040 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.429959    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.516339 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.516310    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.522070 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.521622    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.530797 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.530776    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.631405 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.631364    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.731986 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.731915    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.832434 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.832395    2568 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-255.ec2.internal\" not found"
Jul 28 19:34:46.837791 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.837775    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:34:46.854790 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.854770    2568 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 28 19:34:46.854894 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.854877    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:34:46.854933 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.854911    2568 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 28 19:34:46.892103 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.892082    2568 apiserver.go:52] "Watching apiserver"
Jul 28 19:34:46.912633 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.912053    2568 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 28 19:34:46.912633 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.912512    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.912633 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.912535    2568 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 28 19:34:46.913536 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.913518    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6w927","kube-system/konnectivity-agent-547pv","openshift-cluster-node-tuning-operator/tuned-ww7kh","openshift-dns/node-resolver-jfqbz","openshift-multus/multus-additional-cni-plugins-7x4m5","openshift-multus/multus-dchl6","openshift-network-diagnostics/network-check-target-rh26c","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9","openshift-image-registry/node-ca-mmn7j","openshift-multus/network-metrics-daemon-hs6df","openshift-network-operator/iptables-alerter-4kcj4","openshift-ovn-kubernetes/ovnkube-node-8wcxl"]
Jul 28 19:34:46.916016 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.915997    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:46.918142 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.918124    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:46.918373 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.918349    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 28 19:34:46.918477 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.918354    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-bvgpc\""
Jul 28 19:34:46.918477 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.918357    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:34:46.920268 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.920247    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 28 19:34:46.920268 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.920260    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-cv7g4\""
Jul 28 19:34:46.920532 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.920512    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:46.920608 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.920536    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 28 19:34:46.923140 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.923122    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:46.923233 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.923191    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:34:46.923687 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.923674    2568 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 28 19:34:46.923866 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.923853    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 28 19:34:46.925064 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.925051    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 28 19:34:46.925252 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.925236    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 28 19:34:46.925324 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.925257    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 28 19:34:46.925324 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.925272    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 28 19:34:46.925324 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.925301    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bdmdk\""
Jul 28 19:34:46.925823 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.925807    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:46.927853 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.927833    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 28 19:34:46.927949 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.927915    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 28 19:34:46.928002 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.927965    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mtw9j\""
Jul 28 19:34:46.928110 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.928090    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 28 19:34:46.928433 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.928403    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:46.928520 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.928474    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:34:46.931009 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.930990    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:34:46.931083 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.931067    2568 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal"
Jul 28 19:34:46.931218 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.931201    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:46.933382 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.933363    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 28 19:34:46.933539 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.933522    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 28 19:34:46.933625 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.933598    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-v9rm7\""
Jul 28 19:34:46.933746 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.933731    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 28 19:34:46.933952 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.933932    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:46.934028 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:46.934003    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:34:46.934266 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.934248    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 28 19:34:46.934360 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.934278    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 28 19:34:46.934360 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.934308    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 28 19:34:46.939295 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.939280    2568 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 28 19:34:46.942895 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.942878    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:34:46.945027 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.945011    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 28 19:34:46.945027 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.945020    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 28 19:34:46.945130 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.945038    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-bgntw\""
Jul 28 19:34:46.945502 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.945486    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dchl6"
Jul 28 19:34:46.948359 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.948342    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 28 19:34:46.948481 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.948390    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-k875c\""
Jul 28 19:34:46.949952 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.949936    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:46.950534 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.950516    2568 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cfks7"
Jul 28 19:34:46.951499 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.951459    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-27 19:29:47 +0000 UTC" deadline="2027-12-23 03:46:18.275499493 +0000 UTC"
Jul 28 19:34:46.951616 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.951501    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12296h11m31.324002407s"
Jul 28 19:34:46.952106 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.952088    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 28 19:34:46.952321 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.952306    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 28 19:34:46.952391 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.952335    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 28 19:34:46.952391 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.952311    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wtzh4\""
Jul 28 19:34:46.952627 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.952603    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal"]
Jul 28 19:34:46.952627 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.952644    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal"]
Jul 28 19:34:46.952957 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.952693    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:46.954672 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.954656    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 28 19:34:46.954806 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.954793    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 28 19:34:46.954863 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.954840    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:34:46.954920 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.954905    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-glbbp\""
Jul 28 19:34:46.957646 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.957629    2568 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cfks7"
Jul 28 19:34:46.997409 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:46.997383    2568 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:34:47.013837 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.013814    2568 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 28 19:34:47.019876 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.019856    2568 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:34:47.020122 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020105    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e271958f-3e62-4943-ac71-31f18a0d7369-konnectivity-ca\") pod \"konnectivity-agent-547pv\" (UID: \"e271958f-3e62-4943-ac71-31f18a0d7369\") " pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:34:47.020173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020131    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgrc6\" (UniqueName: \"kubernetes.io/projected/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-kube-api-access-cgrc6\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:47.020173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020149    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adbb15eb-86e8-451d-9cd2-8628d4e85416-tmp\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.020245 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020172    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.020245 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020206    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-ovnkube-config\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.020245 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020225    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9210030c-e254-4197-bf3f-46da9c372a57-multus-daemon-config\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.020245 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020241    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:47.020357 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020262    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66ebf5f5-ab7c-4cb7-9857-b376b202edae-kubelet-config\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:47.020357 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020288    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-sys\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.020357 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020309    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-kubelet\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.020357 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020327    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-cni-netd\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.020493 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020369    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66ebf5f5-ab7c-4cb7-9857-b376b202edae-dbus\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:47.020493 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020401    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysconfig\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.020493 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020441    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-netns\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.020493 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020457    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-etc-kubernetes\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.020493 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020471    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-host\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:47.020493 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020485    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysctl-d\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020511    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-var-lib-kubelet\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020532    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020566    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzlj7\" (UniqueName: \"kubernetes.io/projected/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-kube-api-access-gzlj7\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020585    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-ovn\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020604    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-cni-multus\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020620    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020634    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-tuned\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020648    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-socket-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020673    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-log-socket\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020689    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605d46ac-45a6-48a9-a859-11a690c22efc-tmp-dir\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:47.020710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020711    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-system-cni-dir\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020729    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqtdx\" (UniqueName: \"kubernetes.io/projected/ab602a3a-9515-4404-b7f9-695f78c2b49b-kube-api-access-cqtdx\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020754    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-iptables-alerter-script\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020788    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-kubernetes\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020820    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-run\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020843    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-host\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020871    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020904    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-cni-bin\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020942    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-multus-certs\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020959    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w7jc8\" (UniqueName: \"kubernetes.io/projected/9210030c-e254-4197-bf3f-46da9c372a57-kube-api-access-w7jc8\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020981    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e271958f-3e62-4943-ac71-31f18a0d7369-agent-certs\") pod \"konnectivity-agent-547pv\" (UID: \"e271958f-3e62-4943-ac71-31f18a0d7369\") " pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.020999    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-modprobe-d\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021022    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021044    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-sys-fs\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021058    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-slash\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021076    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-etc-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.021182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021112    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-k8s-cni-cncf-io\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021145    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-kubelet\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021171    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gds9n\" (UniqueName: \"kubernetes.io/projected/adbb15eb-86e8-451d-9cd2-8628d4e85416-kube-api-access-gds9n\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021197    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-registration-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021221    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-conf-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021243    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-cnibin\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021261    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021285    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-cni-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021319    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021344    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/605d46ac-45a6-48a9-a859-11a690c22efc-hosts-file\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021380    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dbz5q\" (UniqueName: \"kubernetes.io/projected/605d46ac-45a6-48a9-a859-11a690c22efc-kube-api-access-dbz5q\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021405    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021442    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-serviceca\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021468    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d25pd\" (UniqueName: \"kubernetes.io/projected/a70f04ad-b295-4af9-8a44-8bf5ff525971-kube-api-access-d25pd\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021490    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-os-release\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021514    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-hostroot\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.021972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021538    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-systemd-units\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021561    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-cnibin\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021582    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-socket-dir-parent\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021606    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-run-netns\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021638    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-env-overrides\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021663    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-ovnkube-script-lib\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021688    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-cni-bin\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021710    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021743    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-host-slash\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021764    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysctl-conf\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021788    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56fc8676-db44-44c6-93ad-5893e67d6386-ovn-node-metrics-cert\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021823    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-lib-modules\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021863    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-device-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021890    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-var-lib-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021918    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021944    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-node-log\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.022551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021966    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-system-cni-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.023047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.021987    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9210030c-e254-4197-bf3f-46da9c372a57-cni-binary-copy\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:47.023047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.022009    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-systemd\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:47.023047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.022031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-systemd\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.023047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.022053    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-run-ovn-kubernetes\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.023047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.022076    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzrkr\" (UniqueName: \"kubernetes.io/projected/56fc8676-db44-44c6-93ad-5893e67d6386-kube-api-access-vzrkr\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:47.023047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.022098    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-os-release\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:47.023047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:47.022122    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grcn7\" (UniqueName: \"kubernetes.io/projected/967eb3cb-a574-4af5-bba1-78655a105b62-kube-api-access-grcn7\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:48.671246 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.671211    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc45489797f52552f43a8393e08ba8c3a.slice/crio-47da49f60c6206afc976f0c65e7a771198ee75d7fb3b7701a04138ff4910e56e WatchSource:0}: Error finding container 47da49f60c6206afc976f0c65e7a771198ee75d7fb3b7701a04138ff4910e56e: Status 404 returned error can't find the container with id 47da49f60c6206afc976f0c65e7a771198ee75d7fb3b7701a04138ff4910e56e
Jul 28 19:34:48.676258 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.676195    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal" event={"ID":"c45489797f52552f43a8393e08ba8c3a","Type":"ContainerStarted","Data":"47da49f60c6206afc976f0c65e7a771198ee75d7fb3b7701a04138ff4910e56e"}
Jul 28 19:34:48.676637 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.676620    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:34:48.681865 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.681837    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1daed9178f1adc845a4a940fe1f2f13f.slice/crio-eddef4b4fa01c4e602ec7e99735d1bada56d56ab7bbb4e042cfa0de0d1e76414 WatchSource:0}: Error finding container eddef4b4fa01c4e602ec7e99735d1bada56d56ab7bbb4e042cfa0de0d1e76414: Status 404 returned error can't find the container with id eddef4b4fa01c4e602ec7e99735d1bada56d56ab7bbb4e042cfa0de0d1e76414
Jul 28 19:34:48.718259 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718235    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-netns\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.718377 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718264    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-etc-kubernetes\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.718377 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718279    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-host\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:48.718377 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718301    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysctl-d\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.718377 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-var-lib-kubelet\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.718377 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718356    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-etc-kubernetes\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718376    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-host\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718389    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-var-lib-kubelet\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718378    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-netns\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718403    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718442    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzlj7\" (UniqueName: \"kubernetes.io/projected/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-kube-api-access-gzlj7\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718460    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysctl-d\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718466    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-ovn\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718503    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-cni-multus\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718507    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-ovn\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:48.718513    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718526    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718547    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-tuned\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718557    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-cni-multus\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:48.718572    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs podName:967eb3cb-a574-4af5-bba1-78655a105b62 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:49.218557545 +0000 UTC m=+2.030119404 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs") pod "network-metrics-daemon-hs6df" (UID: "967eb3cb-a574-4af5-bba1-78655a105b62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718587    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-socket-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718614    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-log-socket\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.718641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718637    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605d46ac-45a6-48a9-a859-11a690c22efc-tmp-dir\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718659    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-system-cni-dir\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718665    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-log-socket\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqtdx\" (UniqueName: \"kubernetes.io/projected/ab602a3a-9515-4404-b7f9-695f78c2b49b-kube-api-access-cqtdx\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718708    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-socket-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718720    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-iptables-alerter-script\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718749    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-kubernetes\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718749    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-system-cni-dir\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718721    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718774    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-run\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718802    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-host\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718823    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718845    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-cni-bin\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718850    2568 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-multus-certs\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718883    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-run\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718893    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w7jc8\" (UniqueName: \"kubernetes.io/projected/9210030c-e254-4197-bf3f-46da9c372a57-kube-api-access-w7jc8\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718900    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-cni-bin\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.719458 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718910    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-host\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718917    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e271958f-3e62-4943-ac71-31f18a0d7369-agent-certs\") pod \"konnectivity-agent-547pv\" (UID: \"e271958f-3e62-4943-ac71-31f18a0d7369\") " pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718922    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-multus-certs\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718864    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-kubernetes\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718945    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-modprobe-d\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718967    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718987    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/605d46ac-45a6-48a9-a859-11a690c22efc-tmp-dir\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718993    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-sys-fs\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719016    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-slash\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719037    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-etc-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719059    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-sys-fs\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719061    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-modprobe-d\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719090    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-k8s-cni-cncf-io\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.718995    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-etc-selinux\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719106    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-slash\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719109    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-kubelet-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719116    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-kubelet\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.720284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719148    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-etc-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719152    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-run-k8s-cni-cncf-io\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719148    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gds9n\" (UniqueName: \"kubernetes.io/projected/adbb15eb-86e8-451d-9cd2-8628d4e85416-kube-api-access-gds9n\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719155    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-kubelet\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719187    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-registration-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719227    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-registration-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719238    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-conf-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719255    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-iptables-alerter-script\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719282    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-cnibin\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719264    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-cnibin\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719311    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-conf-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719346    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-cni-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719402    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719500    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/605d46ac-45a6-48a9-a859-11a690c22efc-hosts-file\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719526    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-cni-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719529    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dbz5q\" (UniqueName: \"kubernetes.io/projected/605d46ac-45a6-48a9-a859-11a690c22efc-kube-api-access-dbz5q\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:48.721117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719573    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/605d46ac-45a6-48a9-a859-11a690c22efc-hosts-file\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719591    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719637    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-serviceca\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719665    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d25pd\" (UniqueName: \"kubernetes.io/projected/a70f04ad-b295-4af9-8a44-8bf5ff525971-kube-api-access-d25pd\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719694    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-os-release\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719717    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-hostroot\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719739    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-systemd-units\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719770    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-cnibin\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719804    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-socket-dir-parent\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719828    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-run-netns\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719852    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-env-overrides\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719876    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-ovnkube-script-lib\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-cni-bin\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719923    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719937    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719946    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-host-slash\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719969    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysctl-conf\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.721964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.719992    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56fc8676-db44-44c6-93ad-5893e67d6386-ovn-node-metrics-cert\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720038    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-lib-modules\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720064    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-device-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720107    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-var-lib-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720109    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-cni-binary-copy\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720132    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720157    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-node-log\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720183    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-serviceca\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720192    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-systemd-units\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720189    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-system-cni-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720231    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-host-var-lib-cni-bin\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720240    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9210030c-e254-4197-bf3f-46da9c372a57-cni-binary-copy\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720266    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-systemd\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720274    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a70f04ad-b295-4af9-8a44-8bf5ff525971-device-dir\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720293    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-systemd\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720337    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-var-lib-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720339    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-lib-modules\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.722787 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720369    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-run-ovn-kubernetes\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720392    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-host-slash\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720395    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzrkr\" (UniqueName: \"kubernetes.io/projected/56fc8676-db44-44c6-93ad-5893e67d6386-kube-api-access-vzrkr\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720438    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-openvswitch\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720394    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-node-log\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720441    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-cnibin\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720454    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-os-release\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720479    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grcn7\" (UniqueName: \"kubernetes.io/projected/967eb3cb-a574-4af5-bba1-78655a105b62-kube-api-access-grcn7\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720485    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-multus-socket-dir-parent\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720509    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e271958f-3e62-4943-ac71-31f18a0d7369-konnectivity-ca\") pod \"konnectivity-agent-547pv\" (UID: \"e271958f-3e62-4943-ac71-31f18a0d7369\") " pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720533    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cgrc6\" (UniqueName: \"kubernetes.io/projected/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-kube-api-access-cgrc6\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720549    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysctl-conf\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720559    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adbb15eb-86e8-451d-9cd2-8628d4e85416-tmp\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:48.720619    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:48.720672    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret podName:66ebf5f5-ab7c-4cb7-9857-b376b202edae nodeName:}" failed. No retries permitted until 2026-07-28 19:34:49.220655549 +0000 UTC m=+2.032217395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret") pod "global-pull-secret-syncer-6w927" (UID: "66ebf5f5-ab7c-4cb7-9857-b376b202edae") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720725    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-ovnkube-script-lib\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720732    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-os-release\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720618    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-ovnkube-config\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720773    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9210030c-e254-4197-bf3f-46da9c372a57-multus-daemon-config\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.720818    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-run-netns\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721085    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-ovnkube-config\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721135    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721182    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/56fc8676-db44-44c6-93ad-5893e67d6386-env-overrides\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721195    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-hostroot\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721220    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721226    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9210030c-e254-4197-bf3f-46da9c372a57-cni-binary-copy\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721362    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66ebf5f5-ab7c-4cb7-9857-b376b202edae-kubelet-config\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721399    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-sys\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721402    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-systemd\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721425    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-kubelet\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721457    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-cni-netd\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721485    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66ebf5f5-ab7c-4cb7-9857-b376b202edae-dbus\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721458    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-run-ovn-kubernetes\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.723802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721561    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysconfig\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721640    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-sysconfig\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721688    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/66ebf5f5-ab7c-4cb7-9857-b376b202edae-dbus\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721723    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-run-systemd\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721773    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ab602a3a-9515-4404-b7f9-695f78c2b49b-os-release\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721809    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9210030c-e254-4197-bf3f-46da9c372a57-multus-daemon-config\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721858    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/adbb15eb-86e8-451d-9cd2-8628d4e85416-sys\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.721934    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ab602a3a-9515-4404-b7f9-695f78c2b49b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.722033    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9210030c-e254-4197-bf3f-46da9c372a57-system-cni-dir\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.722091    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/66ebf5f5-ab7c-4cb7-9857-b376b202edae-kubelet-config\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.722099    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-kubelet\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.722127    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/56fc8676-db44-44c6-93ad-5893e67d6386-host-cni-netd\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.722255    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e271958f-3e62-4943-ac71-31f18a0d7369-konnectivity-ca\") pod \"konnectivity-agent-547pv\" (UID: \"e271958f-3e62-4943-ac71-31f18a0d7369\") " pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.722768    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/adbb15eb-86e8-451d-9cd2-8628d4e85416-etc-tuned\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.722933    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/56fc8676-db44-44c6-93ad-5893e67d6386-ovn-node-metrics-cert\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.722976    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e271958f-3e62-4943-ac71-31f18a0d7369-agent-certs\") pod \"konnectivity-agent-547pv\" (UID: \"e271958f-3e62-4943-ac71-31f18a0d7369\") " pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:34:48.724255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.723013    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/adbb15eb-86e8-451d-9cd2-8628d4e85416-tmp\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.729799 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.729773    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzlj7\" (UniqueName: \"kubernetes.io/projected/40db1398-97f2-4b10-b6b0-debf5bdb6c5d-kube-api-access-gzlj7\") pod \"node-ca-mmn7j\" (UID: \"40db1398-97f2-4b10-b6b0-debf5bdb6c5d\") " pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:48.729893 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.729858    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w7jc8\" (UniqueName: \"kubernetes.io/projected/9210030c-e254-4197-bf3f-46da9c372a57-kube-api-access-w7jc8\") pod \"multus-dchl6\" (UID: \"9210030c-e254-4197-bf3f-46da9c372a57\") " pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.731174 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.731158    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqtdx\" (UniqueName: \"kubernetes.io/projected/ab602a3a-9515-4404-b7f9-695f78c2b49b-kube-api-access-cqtdx\") pod \"multus-additional-cni-plugins-7x4m5\" (UID: \"ab602a3a-9515-4404-b7f9-695f78c2b49b\") " pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.731415 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.731399    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grcn7\" (UniqueName: \"kubernetes.io/projected/967eb3cb-a574-4af5-bba1-78655a105b62-kube-api-access-grcn7\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:48.731485 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:48.731446    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:34:48.731485 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:48.731459    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:34:48.731485 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:48.731469    2568 projected.go:196] Error preparing data for projected volume kube-api-access-hjfpr for pod openshift-network-diagnostics/network-check-target-rh26c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:48.731613 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:48.731516    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr podName:a2f635a2-a6f0-4f62-962f-2d511a939e28 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:49.231504526 +0000 UTC m=+2.043066395 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-hjfpr" (UniqueName: "kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr") pod "network-check-target-rh26c" (UID: "a2f635a2-a6f0-4f62-962f-2d511a939e28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:48.733595 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.733562    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d25pd\" (UniqueName: \"kubernetes.io/projected/a70f04ad-b295-4af9-8a44-8bf5ff525971-kube-api-access-d25pd\") pod \"aws-ebs-csi-driver-node-r6zc9\" (UID: \"a70f04ad-b295-4af9-8a44-8bf5ff525971\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.733751 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.733732    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dbz5q\" (UniqueName: \"kubernetes.io/projected/605d46ac-45a6-48a9-a859-11a690c22efc-kube-api-access-dbz5q\") pod \"node-resolver-jfqbz\" (UID: \"605d46ac-45a6-48a9-a859-11a690c22efc\") " pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:48.734084 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.734068    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgrc6\" (UniqueName: \"kubernetes.io/projected/ab54f959-5cf7-4e9b-9514-9b5fd11a6e49-kube-api-access-cgrc6\") pod \"iptables-alerter-4kcj4\" (UID: \"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49\") " pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:48.734637 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.734621    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzrkr\" (UniqueName: \"kubernetes.io/projected/56fc8676-db44-44c6-93ad-5893e67d6386-kube-api-access-vzrkr\") pod \"ovnkube-node-8wcxl\" (UID: \"56fc8676-db44-44c6-93ad-5893e67d6386\") " pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.734682 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.734634    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gds9n\" (UniqueName: \"kubernetes.io/projected/adbb15eb-86e8-451d-9cd2-8628d4e85416-kube-api-access-gds9n\") pod \"tuned-ww7kh\" (UID: \"adbb15eb-86e8-451d-9cd2-8628d4e85416\") " pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.833069 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.833034    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-ww7kh"
Jul 28 19:34:48.842458 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.842436    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podadbb15eb_86e8_451d_9cd2_8628d4e85416.slice/crio-c4b4d84f349ccb2aebd21f2c134af544554aaeb88ee996684c3eb4710547f2fd WatchSource:0}: Error finding container c4b4d84f349ccb2aebd21f2c134af544554aaeb88ee996684c3eb4710547f2fd: Status 404 returned error can't find the container with id c4b4d84f349ccb2aebd21f2c134af544554aaeb88ee996684c3eb4710547f2fd
Jul 28 19:34:48.870494 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.870467    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-jfqbz"
Jul 28 19:34:48.884847 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.884819    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7x4m5"
Jul 28 19:34:48.894275 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.894249    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab602a3a_9515_4404_b7f9_695f78c2b49b.slice/crio-6150d2b23c1345116162dbfd5997c207ba50ef907cd10051a8cd0c76eb2bea53 WatchSource:0}: Error finding container 6150d2b23c1345116162dbfd5997c207ba50ef907cd10051a8cd0c76eb2bea53: Status 404 returned error can't find the container with id 6150d2b23c1345116162dbfd5997c207ba50ef907cd10051a8cd0c76eb2bea53
Jul 28 19:34:48.897721 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.897685    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9"
Jul 28 19:34:48.906583 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.906561    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda70f04ad_b295_4af9_8a44_8bf5ff525971.slice/crio-afd311298956517ce257363b448dbafb9b62e9df033b12ceee3a6fa35820d8e4 WatchSource:0}: Error finding container afd311298956517ce257363b448dbafb9b62e9df033b12ceee3a6fa35820d8e4: Status 404 returned error can't find the container with id afd311298956517ce257363b448dbafb9b62e9df033b12ceee3a6fa35820d8e4
Jul 28 19:34:48.911612 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.911593    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:34:48.919109 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.919084    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56fc8676_db44_44c6_93ad_5893e67d6386.slice/crio-e7aab882e1a7f33275c284fc4ac9c05cc22d4f93db76433f6529b3618e3c77d2 WatchSource:0}: Error finding container e7aab882e1a7f33275c284fc4ac9c05cc22d4f93db76433f6529b3618e3c77d2: Status 404 returned error can't find the container with id e7aab882e1a7f33275c284fc4ac9c05cc22d4f93db76433f6529b3618e3c77d2
Jul 28 19:34:48.924241 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.924221    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:34:48.931986 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.931966    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode271958f_3e62_4943_ac71_31f18a0d7369.slice/crio-3581d8877c20d88c219d4756658d3407a262ddb8c5d6831723f2bfa8af95a361 WatchSource:0}: Error finding container 3581d8877c20d88c219d4756658d3407a262ddb8c5d6831723f2bfa8af95a361: Status 404 returned error can't find the container with id 3581d8877c20d88c219d4756658d3407a262ddb8c5d6831723f2bfa8af95a361
Jul 28 19:34:48.932075 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.932057    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-dchl6"
Jul 28 19:34:48.939944 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.939925    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9210030c_e254_4197_bf3f_46da9c372a57.slice/crio-d367febbaeb67f56321f2a8a8b6b53ecdd65fd36a29c6d43245389001e7b6347 WatchSource:0}: Error finding container d367febbaeb67f56321f2a8a8b6b53ecdd65fd36a29c6d43245389001e7b6347: Status 404 returned error can't find the container with id d367febbaeb67f56321f2a8a8b6b53ecdd65fd36a29c6d43245389001e7b6347
Jul 28 19:34:48.948294 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.948279    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mmn7j"
Jul 28 19:34:48.952769 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:48.952751    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4kcj4"
Jul 28 19:34:48.956829 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.956810    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40db1398_97f2_4b10_b6b0_debf5bdb6c5d.slice/crio-5682a0130ceea6d94a86f05fa7f35b3732adeb011748d30a4c2636b82e15d529 WatchSource:0}: Error finding container 5682a0130ceea6d94a86f05fa7f35b3732adeb011748d30a4c2636b82e15d529: Status 404 returned error can't find the container with id 5682a0130ceea6d94a86f05fa7f35b3732adeb011748d30a4c2636b82e15d529
Jul 28 19:34:48.961601 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:34:48.961576    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab54f959_5cf7_4e9b_9514_9b5fd11a6e49.slice/crio-6fb9335f4f819bed2f1313ca2036a41721a7ba490197db37740e64b73e2d2d92 WatchSource:0}: Error finding container 6fb9335f4f819bed2f1313ca2036a41721a7ba490197db37740e64b73e2d2d92: Status 404 returned error can't find the container with id 6fb9335f4f819bed2f1313ca2036a41721a7ba490197db37740e64b73e2d2d92
Jul 28 19:34:49.225888 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.225808    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:49.226030 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.225886    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:49.226127 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:49.226032    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:49.226127 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:49.226093    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret podName:66ebf5f5-ab7c-4cb7-9857-b376b202edae nodeName:}" failed. No retries permitted until 2026-07-28 19:34:50.226075644 +0000 UTC m=+3.037637486 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret") pod "global-pull-secret-syncer-6w927" (UID: "66ebf5f5-ab7c-4cb7-9857-b376b202edae") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:49.226522 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:49.226502    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:49.226600 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:49.226557    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs podName:967eb3cb-a574-4af5-bba1-78655a105b62 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:50.226541859 +0000 UTC m=+3.038103716 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs") pod "network-metrics-daemon-hs6df" (UID: "967eb3cb-a574-4af5-bba1-78655a105b62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:49.326849 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.326762    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:49.327018 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:49.326981    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:34:49.327018 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:49.327007    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:34:49.327133 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:49.327021    2568 projected.go:196] Error preparing data for projected volume kube-api-access-hjfpr for pod openshift-network-diagnostics/network-check-target-rh26c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:49.327133 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:49.327089    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr podName:a2f635a2-a6f0-4f62-962f-2d511a939e28 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:50.327068747 +0000 UTC m=+3.138630602 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-hjfpr" (UniqueName: "kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr") pod "network-check-target-rh26c" (UID: "a2f635a2-a6f0-4f62-962f-2d511a939e28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:49.413368 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.413151    2568 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:34:49.555771 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.555587    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:29:48 +0000 UTC" deadline="2028-01-31 11:56:26.199895448 +0000 UTC"
Jul 28 19:34:49.555771 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.555644    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13240h21m36.644255397s"
Jul 28 19:34:49.719651 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.719610    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" event={"ID":"ab602a3a-9515-4404-b7f9-695f78c2b49b","Type":"ContainerStarted","Data":"6150d2b23c1345116162dbfd5997c207ba50ef907cd10051a8cd0c76eb2bea53"}
Jul 28 19:34:49.742165 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.742127    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jfqbz" event={"ID":"605d46ac-45a6-48a9-a859-11a690c22efc","Type":"ContainerStarted","Data":"7373393fbf270c19627b269b1f62799d9ca16febcda56be5249012bfddd5b0b6"}
Jul 28 19:34:49.760242 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.760206    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ww7kh" event={"ID":"adbb15eb-86e8-451d-9cd2-8628d4e85416","Type":"ContainerStarted","Data":"c4b4d84f349ccb2aebd21f2c134af544554aaeb88ee996684c3eb4710547f2fd"}
Jul 28 19:34:49.773841 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.773806    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4kcj4" event={"ID":"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49","Type":"ContainerStarted","Data":"6fb9335f4f819bed2f1313ca2036a41721a7ba490197db37740e64b73e2d2d92"}
Jul 28 19:34:49.785800 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.785760    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mmn7j" event={"ID":"40db1398-97f2-4b10-b6b0-debf5bdb6c5d","Type":"ContainerStarted","Data":"5682a0130ceea6d94a86f05fa7f35b3732adeb011748d30a4c2636b82e15d529"}
Jul 28 19:34:49.791583 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.791550    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dchl6" event={"ID":"9210030c-e254-4197-bf3f-46da9c372a57","Type":"ContainerStarted","Data":"d367febbaeb67f56321f2a8a8b6b53ecdd65fd36a29c6d43245389001e7b6347"}
Jul 28 19:34:49.808535 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.808432    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal" event={"ID":"1daed9178f1adc845a4a940fe1f2f13f","Type":"ContainerStarted","Data":"eddef4b4fa01c4e602ec7e99735d1bada56d56ab7bbb4e042cfa0de0d1e76414"}
Jul 28 19:34:49.817127 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.817095    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-547pv" event={"ID":"e271958f-3e62-4943-ac71-31f18a0d7369","Type":"ContainerStarted","Data":"3581d8877c20d88c219d4756658d3407a262ddb8c5d6831723f2bfa8af95a361"}
Jul 28 19:34:49.832093 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.832059    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"e7aab882e1a7f33275c284fc4ac9c05cc22d4f93db76433f6529b3618e3c77d2"}
Jul 28 19:34:49.848012 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:49.847980    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9" event={"ID":"a70f04ad-b295-4af9-8a44-8bf5ff525971","Type":"ContainerStarted","Data":"afd311298956517ce257363b448dbafb9b62e9df033b12ceee3a6fa35820d8e4"}
Jul 28 19:34:50.122606 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.122528    2568 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 28 19:34:50.236777 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.236743    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:50.236947 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.236822    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:50.237010 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.236953    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:50.237068 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.237018    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret podName:66ebf5f5-ab7c-4cb7-9857-b376b202edae nodeName:}" failed. No retries permitted until 2026-07-28 19:34:52.236996742 +0000 UTC m=+5.048558600 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret") pod "global-pull-secret-syncer-6w927" (UID: "66ebf5f5-ab7c-4cb7-9857-b376b202edae") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:50.237127 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.237090    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:50.237127 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.237124    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs podName:967eb3cb-a574-4af5-bba1-78655a105b62 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:52.237113156 +0000 UTC m=+5.048675010 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs") pod "network-metrics-daemon-hs6df" (UID: "967eb3cb-a574-4af5-bba1-78655a105b62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:50.337424 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.337369    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:50.337611 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.337528    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:34:50.337611 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.337550    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:34:50.337611 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.337564    2568 projected.go:196] Error preparing data for projected volume kube-api-access-hjfpr for pod openshift-network-diagnostics/network-check-target-rh26c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:50.337817 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.337617    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr podName:a2f635a2-a6f0-4f62-962f-2d511a939e28 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:52.337599924 +0000 UTC m=+5.149161769 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-hjfpr" (UniqueName: "kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr") pod "network-check-target-rh26c" (UID: "a2f635a2-a6f0-4f62-962f-2d511a939e28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:50.556215 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.556127    2568 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-27 19:29:48 +0000 UTC" deadline="2028-01-03 03:22:37.259414842 +0000 UTC"
Jul 28 19:34:50.556215 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.556167    2568 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12559h47m46.703251873s"
Jul 28 19:34:50.670017 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.669988    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:50.670017 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.669998    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:50.670235 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:50.669986    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:50.670235 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.670113    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:34:50.670235 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.670163    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:34:50.670408 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:50.670262    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:34:52.255867 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:52.255080    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:52.255867 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:52.255149    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:52.255867 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.255273    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:52.255867 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.255346    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs podName:967eb3cb-a574-4af5-bba1-78655a105b62 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:56.255310757 +0000 UTC m=+9.066872611 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs") pod "network-metrics-daemon-hs6df" (UID: "967eb3cb-a574-4af5-bba1-78655a105b62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:52.255867 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.255763    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:52.255867 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.255822    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret podName:66ebf5f5-ab7c-4cb7-9857-b376b202edae nodeName:}" failed. No retries permitted until 2026-07-28 19:34:56.255806408 +0000 UTC m=+9.067368266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret") pod "global-pull-secret-syncer-6w927" (UID: "66ebf5f5-ab7c-4cb7-9857-b376b202edae") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:52.356046 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:52.355956    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:52.356203 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.356168    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:34:52.356203 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.356189    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:34:52.356203 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.356202    2568 projected.go:196] Error preparing data for projected volume kube-api-access-hjfpr for pod openshift-network-diagnostics/network-check-target-rh26c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:52.356394 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.356256    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr podName:a2f635a2-a6f0-4f62-962f-2d511a939e28 nodeName:}" failed. No retries permitted until 2026-07-28 19:34:56.356239409 +0000 UTC m=+9.167801252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-hjfpr" (UniqueName: "kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr") pod "network-check-target-rh26c" (UID: "a2f635a2-a6f0-4f62-962f-2d511a939e28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:52.670265 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:52.670181    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:52.670424 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.670303    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:34:52.670424 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:52.670391    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:52.670546 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.670480    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:34:52.670546 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:52.670543    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:52.670651 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:52.670607    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:34:54.670386 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:54.670287    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:54.670386 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:54.670340    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:54.670891 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:54.670452    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:34:54.670891 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:54.670539    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:34:54.670891 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:54.670591    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:54.670891 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:54.670662    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:34:56.291331 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:56.290976    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:56.291331 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:56.291045    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:56.291331 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.291174    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:56.291331 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.291190    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:56.291331 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.291252    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret podName:66ebf5f5-ab7c-4cb7-9857-b376b202edae nodeName:}" failed. No retries permitted until 2026-07-28 19:35:04.291227166 +0000 UTC m=+17.102789023 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret") pod "global-pull-secret-syncer-6w927" (UID: "66ebf5f5-ab7c-4cb7-9857-b376b202edae") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:34:56.291331 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.291271    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs podName:967eb3cb-a574-4af5-bba1-78655a105b62 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:04.291261589 +0000 UTC m=+17.102823447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs") pod "network-metrics-daemon-hs6df" (UID: "967eb3cb-a574-4af5-bba1-78655a105b62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:34:56.391453 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:56.391423    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:56.391665 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.391634    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:34:56.391665 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.391666    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:34:56.391859 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.391681    2568 projected.go:196] Error preparing data for projected volume kube-api-access-hjfpr for pod openshift-network-diagnostics/network-check-target-rh26c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:56.391859 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.391745    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr podName:a2f635a2-a6f0-4f62-962f-2d511a939e28 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:04.391726963 +0000 UTC m=+17.203288819 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-hjfpr" (UniqueName: "kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr") pod "network-check-target-rh26c" (UID: "a2f635a2-a6f0-4f62-962f-2d511a939e28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:34:56.670709 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:56.670629    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:56.670709 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:56.670658    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:56.670913 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:56.670629    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:56.670913 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.670746    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:34:56.670913 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.670848    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:34:56.671066 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:56.670975    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:34:58.670533 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:58.670493    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:34:58.670533 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:58.670524    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:34:58.670943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:34:58.670524    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:34:58.670943 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:58.670651    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:34:58.670943 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:58.670680    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:34:58.670943 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:34:58.670740    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:00.670751 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:00.670718    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:00.670751 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:00.670748    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:00.670751 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:00.670718    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:00.671277 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:00.670848    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:00.671277 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:00.671001    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:00.671277 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:00.671103    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:02.670205 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:02.670164    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:02.670205 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:02.670194    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:02.670732 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:02.670278    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:02.670732 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:02.670291    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:02.670732 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:02.670401    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:02.670732 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:02.670521    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:04.350457 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:04.350417    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:04.350906 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:04.350493    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:04.350906 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.350568    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:04.350906 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.350638    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs podName:967eb3cb-a574-4af5-bba1-78655a105b62 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:20.35061986 +0000 UTC m=+33.162181706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs") pod "network-metrics-daemon-hs6df" (UID: "967eb3cb-a574-4af5-bba1-78655a105b62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:04.350906 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.350647    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:04.350906 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.350701    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret podName:66ebf5f5-ab7c-4cb7-9857-b376b202edae nodeName:}" failed. No retries permitted until 2026-07-28 19:35:20.350683119 +0000 UTC m=+33.162244974 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret") pod "global-pull-secret-syncer-6w927" (UID: "66ebf5f5-ab7c-4cb7-9857-b376b202edae") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:04.451203 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:04.451165    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:04.451352 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.451306    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:04.451352 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.451338    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:04.451352 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.451347    2568 projected.go:196] Error preparing data for projected volume kube-api-access-hjfpr for pod openshift-network-diagnostics/network-check-target-rh26c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:04.451490 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.451401    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr podName:a2f635a2-a6f0-4f62-962f-2d511a939e28 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:20.451384077 +0000 UTC m=+33.262945924 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-hjfpr" (UniqueName: "kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr") pod "network-check-target-rh26c" (UID: "a2f635a2-a6f0-4f62-962f-2d511a939e28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:04.670250 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:04.670163    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:04.670435 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:04.670163    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:04.670435 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.670285    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:04.670435 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.670406    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:04.670435 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:04.670164    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:04.670635 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:04.670543    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:06.670972 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:06.670745    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:06.671463 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:06.670851    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:06.671463 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:06.671083    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:06.671463 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:06.670916    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:06.671463 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:06.671231    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:06.671463 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:06.671236    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:08.670013 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:08.669992    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:08.670365 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:08.670096    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:08.670365 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:08.670102    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:08.670365 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:08.670134    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:08.670365 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:08.670240    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:08.670511 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:08.670359    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:08.891590 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:08.891562    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-ww7kh" event={"ID":"adbb15eb-86e8-451d-9cd2-8628d4e85416","Type":"ContainerStarted","Data":"b1f39a7d3e67e758c30722f093fd320bd5d81a511478d739301e9ba5d50a6d89"}
Jul 28 19:35:08.892927 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:08.892895    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal" event={"ID":"c45489797f52552f43a8393e08ba8c3a","Type":"ContainerStarted","Data":"68ad6a86fd5cbe5f7a491ab64b35df34c84334a22d5602486ce1d827fc2c21b9"}
Jul 28 19:35:08.894241 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:08.894220    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-dchl6" event={"ID":"9210030c-e254-4197-bf3f-46da9c372a57","Type":"ContainerStarted","Data":"538ff6aa622f70a102d6c76026eaf77997c872a01b775d9c09138f353d97d236"}
Jul 28 19:35:08.908142 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:08.908071    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-255.ec2.internal" podStartSLOduration=20.908059121 podStartE2EDuration="20.908059121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:08.907667634 +0000 UTC m=+21.719229497" watchObservedRunningTime="2026-07-28 19:35:08.908059121 +0000 UTC m=+21.719620983"
Jul 28 19:35:09.898495 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.898231    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mmn7j" event={"ID":"40db1398-97f2-4b10-b6b0-debf5bdb6c5d","Type":"ContainerStarted","Data":"7d66b6ccac7b385f923bb5bc995f9f0ce77248a9e910465e24191dfae2ac2361"}
Jul 28 19:35:09.899928 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.899900    2568 generic.go:356] "Generic (PLEG): container finished" podID="1daed9178f1adc845a4a940fe1f2f13f" containerID="9edbe7ac68439e647de15680bb84a2b507d44664381bc689baad5107908a4264" exitCode=0
Jul 28 19:35:09.900059 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.899984    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal" event={"ID":"1daed9178f1adc845a4a940fe1f2f13f","Type":"ContainerDied","Data":"9edbe7ac68439e647de15680bb84a2b507d44664381bc689baad5107908a4264"}
Jul 28 19:35:09.902084 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.902057    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-547pv" event={"ID":"e271958f-3e62-4943-ac71-31f18a0d7369","Type":"ContainerStarted","Data":"9784bd08c974cd92ee61488398cbb10f9d789d06247d464c3b1863d6bf9db6c9"}
Jul 28 19:35:09.905607 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.905574    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"02789f43091b3cbc8cfdbe276aec0f529d8f8a5264d98cb84b9584c30b91b071"}
Jul 28 19:35:09.905708 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.905658    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"59eb970b2895c181657345c3fca0023e71e426ae844f073d3e4a3d9bf7d7569e"}
Jul 28 19:35:09.905774 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.905708    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"bff9a3b41e3db7288dc8ddac40646011771ce767c70a302cb270c43782aaa880"}
Jul 28 19:35:09.905774 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.905758    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"1b0babb4dfb1e6c24f8e5d4d8e76f22a5c47d61c7e5d66a2a86aeb52b7dd74ab"}
Jul 28 19:35:09.905858 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.905791    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"1fd0bb98637285e9b39e73936d48020f5c31208f3381b635e34e3ae34305a0c0"}
Jul 28 19:35:09.905858 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.905823    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"eaa50422c705dd39e7e686bfe43af41e439dacb3360ab2f80ecbb0281e70a9ae"}
Jul 28 19:35:09.907278 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.907201    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9" event={"ID":"a70f04ad-b295-4af9-8a44-8bf5ff525971","Type":"ContainerStarted","Data":"9aa625b629f52eb072b0da60b0435c2aa982275d2319215058c8f5e5da554036"}
Jul 28 19:35:09.908753 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.908728    2568 generic.go:356] "Generic (PLEG): container finished" podID="ab602a3a-9515-4404-b7f9-695f78c2b49b" containerID="ebd353079f5fedbaeeb20aa3395705966969c97fae56f408dcddbc382e326933" exitCode=0
Jul 28 19:35:09.908860 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.908810    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" event={"ID":"ab602a3a-9515-4404-b7f9-695f78c2b49b","Type":"ContainerDied","Data":"ebd353079f5fedbaeeb20aa3395705966969c97fae56f408dcddbc382e326933"}
Jul 28 19:35:09.910424 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.910399    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-jfqbz" event={"ID":"605d46ac-45a6-48a9-a859-11a690c22efc","Type":"ContainerStarted","Data":"50d68cf2a5758e4800c58725696a6cf9504ab15ace526e76df32404933e4d54b"}
Jul 28 19:35:09.912336 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.912279    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-dchl6" podStartSLOduration=2.283482004 podStartE2EDuration="21.912251684s" totalImagesPullingTime="19.62876968s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:08.924132433 +0000 UTC m=+21.735694297" watchObservedRunningTime="2026-07-28 19:35:09.912251684 +0000 UTC m=+22.723813548"
Jul 28 19:35:09.912548 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.912524    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mmn7j" podStartSLOduration=2.302234158 podStartE2EDuration="21.912519375s" totalImagesPullingTime="19.610285217s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:09.912205078 +0000 UTC m=+22.723766941" watchObservedRunningTime="2026-07-28 19:35:09.912519375 +0000 UTC m=+22.724081238"
Jul 28 19:35:09.952686 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.952431    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-547pv" podStartSLOduration=2.317495267 podStartE2EDuration="21.952411696s" totalImagesPullingTime="19.634916429s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:09.951804407 +0000 UTC m=+22.763366295" watchObservedRunningTime="2026-07-28 19:35:09.952411696 +0000 UTC m=+22.763973560"
Jul 28 19:35:09.981058 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.981007    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-ww7kh" podStartSLOduration=3.254011897 podStartE2EDuration="22.980988292s" totalImagesPullingTime="19.726976395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:09.980949441 +0000 UTC m=+22.792511305" watchObservedRunningTime="2026-07-28 19:35:09.980988292 +0000 UTC m=+22.792550155"
Jul 28 19:35:09.994768 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:09.994220    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-jfqbz" podStartSLOduration=3.337874752 podStartE2EDuration="22.994202045s" totalImagesPullingTime="19.656327293s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:09.994123314 +0000 UTC m=+22.805685178" watchObservedRunningTime="2026-07-28 19:35:09.994202045 +0000 UTC m=+22.805763912"
Jul 28 19:35:10.669980 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:10.669952    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:10.670124 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:10.669969    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:10.670124 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:10.670041    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:10.670124 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:10.670058    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:10.670124 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:10.670105    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:10.670348 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:10.670188    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:10.744635 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:10.744608    2568 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 28 19:35:10.915086 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:10.914979    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9" event={"ID":"a70f04ad-b295-4af9-8a44-8bf5ff525971","Type":"ContainerStarted","Data":"03c93af54c915123a3916695f20daf130eecd841060d05d424d122309a8d2d47"}
Jul 28 19:35:10.916729 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:10.916704    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4kcj4" event={"ID":"ab54f959-5cf7-4e9b-9514-9b5fd11a6e49","Type":"ContainerStarted","Data":"8cd775863d40f636ca0cb81cff976abf216babdb0892fdab91f24fd49fd3b213"}
Jul 28 19:35:10.918626 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:10.918599    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal" event={"ID":"1daed9178f1adc845a4a940fe1f2f13f","Type":"ContainerStarted","Data":"4beff64b6b34cf53123e69b86f1d9027b206e16b7f237c1339d8aca20b521b15"}
Jul 28 19:35:10.931094 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:10.931054    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4kcj4" podStartSLOduration=3.322949759 podStartE2EDuration="22.931039708s" totalImagesPullingTime="19.608089949s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:10.930813337 +0000 UTC m=+23.742375200" watchObservedRunningTime="2026-07-28 19:35:10.931039708 +0000 UTC m=+23.742601570"
Jul 28 19:35:11.217375 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:11.217287    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:35:11.217983 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:11.217956    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:35:11.233694 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:11.233652    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-255.ec2.internal" podStartSLOduration=23.233634827 podStartE2EDuration="23.233634827s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:10.946540581 +0000 UTC m=+23.758102470" watchObservedRunningTime="2026-07-28 19:35:11.233634827 +0000 UTC m=+24.045196704"
Jul 28 19:35:11.580772 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:11.580615    2568 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-28T19:35:10.744628655Z","UUID":"0c094105-2dfb-4fce-acb1-d40ffc23fc17","Handler":null,"Name":"","Endpoint":""}
Jul 28 19:35:11.582381 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:11.582359    2568 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 28 19:35:11.582495 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:11.582399    2568 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 28 19:35:11.932975 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:11.932951    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:35:11.933536 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:11.933517    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-547pv"
Jul 28 19:35:12.670278 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:12.670241    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:12.670461 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:12.670286    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:12.670461 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:12.670286    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:12.670461 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:12.670388    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:12.670461 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:12.670446    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:12.670609 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:12.670563    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:12.926709 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:12.926630    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"8906aab072a667e11f3a481412ee6ed6ce683d3d5abb3e5b48712144629cbd2e"}
Jul 28 19:35:12.928932 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:12.928880    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9" event={"ID":"a70f04ad-b295-4af9-8a44-8bf5ff525971","Type":"ContainerStarted","Data":"c96236b6b64acb6c22572887f315a452910ad0f612b5708b6314377944b8b36d"}
Jul 28 19:35:12.946159 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:12.946116    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-r6zc9" podStartSLOduration=4.013582972 podStartE2EDuration="25.946103988s" totalImagesPullingTime="21.932521016s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:47 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:12.94572658 +0000 UTC m=+25.757288477" watchObservedRunningTime="2026-07-28 19:35:12.946103988 +0000 UTC m=+25.757665847"
Jul 28 19:35:14.670626 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.670445    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:14.671307 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.670447    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:14.671307 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:14.670694    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:14.671307 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:14.670795    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:14.671307 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.670544    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:14.671307 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:14.670878    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:14.936439 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.936341    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" event={"ID":"56fc8676-db44-44c6-93ad-5893e67d6386","Type":"ContainerStarted","Data":"7dc3ab158850e727f9f97b434d2df152d535162386c3f24e50740ae03c660303"}
Jul 28 19:35:14.936605 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.936481    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:35:14.936605 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.936539    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:35:14.938278 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.938247    2568 generic.go:356] "Generic (PLEG): container finished" podID="ab602a3a-9515-4404-b7f9-695f78c2b49b" containerID="a28ce1a2ca7d83eb7e2f96379fde50f474dea43ec68c64681ea109de6564e1fc" exitCode=0
Jul 28 19:35:14.938411 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.938292    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" event={"ID":"ab602a3a-9515-4404-b7f9-695f78c2b49b","Type":"ContainerDied","Data":"a28ce1a2ca7d83eb7e2f96379fde50f474dea43ec68c64681ea109de6564e1fc"}
Jul 28 19:35:14.952435 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.952416    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:35:14.962837 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:14.962801    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl" podStartSLOduration=7.823551722 podStartE2EDuration="27.962787751s" totalImagesPullingTime="20.139236029s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:14.961260186 +0000 UTC m=+27.772822071" watchObservedRunningTime="2026-07-28 19:35:14.962787751 +0000 UTC m=+27.774349614"
Jul 28 19:35:15.941173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:15.941046    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:35:15.960407 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:15.960379    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:35:16.156923 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:16.156885    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rh26c"]
Jul 28 19:35:16.157132 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:16.157060    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:16.157221 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:16.157159    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:16.160620 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:16.160567    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6w927"]
Jul 28 19:35:16.160735 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:16.160712    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:16.160854 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:16.160833    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:16.161158 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:16.161134    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hs6df"]
Jul 28 19:35:16.161280 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:16.161235    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:16.161373 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:16.161349    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:16.944886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:16.944854    2568 generic.go:356] "Generic (PLEG): container finished" podID="ab602a3a-9515-4404-b7f9-695f78c2b49b" containerID="c2079e9c78b268b69aa119f1f0766e0a9451b13d4c01bc796f2653e1a5c6e7de" exitCode=0
Jul 28 19:35:16.945256 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:16.944935    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" event={"ID":"ab602a3a-9515-4404-b7f9-695f78c2b49b","Type":"ContainerDied","Data":"c2079e9c78b268b69aa119f1f0766e0a9451b13d4c01bc796f2653e1a5c6e7de"}
Jul 28 19:35:17.671313 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:17.671281    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:17.671513 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:17.671391    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:17.671513 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:17.671433    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:17.671587 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:17.671518    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:17.671587 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:17.671551    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:17.671654 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:17.671635    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:18.951467 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:18.951434    2568 generic.go:356] "Generic (PLEG): container finished" podID="ab602a3a-9515-4404-b7f9-695f78c2b49b" containerID="d23daeb6017ef0d79bf263f87aa595a0d0aa40721210bd0b69a56494980ff810" exitCode=0
Jul 28 19:35:18.952020 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:18.951486    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" event={"ID":"ab602a3a-9515-4404-b7f9-695f78c2b49b","Type":"ContainerDied","Data":"d23daeb6017ef0d79bf263f87aa595a0d0aa40721210bd0b69a56494980ff810"}
Jul 28 19:35:19.670227 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:19.670187    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:19.670432 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:19.670313    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:19.670432 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:19.670338    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:19.670547 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:19.670453    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:19.670547 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:19.670477    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:19.670639 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:19.670606    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:20.377410 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:20.377371    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:20.377812 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:20.377468    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:20.377812 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:20.377532    2568 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:20.377812 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:20.377579    2568 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:20.377812 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:20.377606    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs podName:967eb3cb-a574-4af5-bba1-78655a105b62 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:52.377589263 +0000 UTC m=+65.189151107 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs") pod "network-metrics-daemon-hs6df" (UID: "967eb3cb-a574-4af5-bba1-78655a105b62") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 28 19:35:20.377812 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:20.377626    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret podName:66ebf5f5-ab7c-4cb7-9857-b376b202edae nodeName:}" failed. No retries permitted until 2026-07-28 19:35:52.377612278 +0000 UTC m=+65.189174144 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret") pod "global-pull-secret-syncer-6w927" (UID: "66ebf5f5-ab7c-4cb7-9857-b376b202edae") : object "kube-system"/"original-pull-secret" not registered
Jul 28 19:35:20.478731 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:20.478695    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:20.478908 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:20.478834    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 28 19:35:20.478908 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:20.478859    2568 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 28 19:35:20.478908 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:20.478874    2568 projected.go:196] Error preparing data for projected volume kube-api-access-hjfpr for pod openshift-network-diagnostics/network-check-target-rh26c: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:20.479026 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:20.478944    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr podName:a2f635a2-a6f0-4f62-962f-2d511a939e28 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:52.47892303 +0000 UTC m=+65.290484886 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-hjfpr" (UniqueName: "kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr") pod "network-check-target-rh26c" (UID: "a2f635a2-a6f0-4f62-962f-2d511a939e28") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 28 19:35:21.670709 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:21.670664    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:21.671164 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:21.670677    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:21.671164 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:21.670800    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-rh26c" podUID="a2f635a2-a6f0-4f62-962f-2d511a939e28"
Jul 28 19:35:21.671164 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:21.670683    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:21.671164 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:21.670881    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-hs6df" podUID="967eb3cb-a574-4af5-bba1-78655a105b62"
Jul 28 19:35:21.671164 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:21.670959    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-6w927" podUID="66ebf5f5-ab7c-4cb7-9857-b376b202edae"
Jul 28 19:35:22.554651 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.554613    2568 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-255.ec2.internal" event="NodeReady"
Jul 28 19:35:22.554813 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.554782    2568 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 28 19:35:22.595398 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.595367    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6db6c45f8b-r74tv"]
Jul 28 19:35:22.628771 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.628730    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6db6c45f8b-r74tv"]
Jul 28 19:35:22.628771 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.628777    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gzdh7"]
Jul 28 19:35:22.629004 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.628871    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.632657 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.631351    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9p5kr\""
Jul 28 19:35:22.632657 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.631609    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Jul 28 19:35:22.632657 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.631801    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Jul 28 19:35:22.632657 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.632555    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Jul 28 19:35:22.637503 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.637480    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Jul 28 19:35:22.643668 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.643645    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-wkdkt"]
Jul 28 19:35:22.643779 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.643761    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.646544 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.646483    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lt6xm\""
Jul 28 19:35:22.646544 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.646516    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 28 19:35:22.646685 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.646484    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 28 19:35:22.661670 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.661652    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wkdkt"]
Jul 28 19:35:22.661768 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.661680    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gzdh7"]
Jul 28 19:35:22.661825 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.661775    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wkdkt"
Jul 28 19:35:22.664640 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.664612    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 28 19:35:22.665304 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.665267    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 28 19:35:22.665855 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.665836    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 28 19:35:22.666026 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.664612    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cpl6j\""
Jul 28 19:35:22.796642 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796559    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-certificates\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.796642 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796613    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac898250-c4c6-4362-b0c6-f3149a5f299f-cert\") pod \"ingress-canary-wkdkt\" (UID: \"ac898250-c4c6-4362-b0c6-f3149a5f299f\") " pod="openshift-ingress-canary/ingress-canary-wkdkt"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796698    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-bound-sa-token\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796734    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796770    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-trusted-ca\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796802    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce1caf2-c10b-4415-9058-7688ea24cafd-config-volume\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796819    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khttl\" (UniqueName: \"kubernetes.io/projected/9ce1caf2-c10b-4415-9058-7688ea24cafd-kube-api-access-khttl\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796850    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796870    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-installation-pull-secrets\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796917    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jn6c8\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-kube-api-access-jn6c8\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796955    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ce1caf2-c10b-4415-9058-7688ea24cafd-tmp-dir\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.796992    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-image-registry-private-configuration\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.797043    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-ca-trust-extracted\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.797109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.797072    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6ls2w\" (UniqueName: \"kubernetes.io/projected/ac898250-c4c6-4362-b0c6-f3149a5f299f-kube-api-access-6ls2w\") pod \"ingress-canary-wkdkt\" (UID: \"ac898250-c4c6-4362-b0c6-f3149a5f299f\") " pod="openshift-ingress-canary/ingress-canary-wkdkt"
Jul 28 19:35:22.897880 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.897846    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-image-registry-private-configuration\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898040 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.897900    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-ca-trust-extracted\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898040 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.897925    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6ls2w\" (UniqueName: \"kubernetes.io/projected/ac898250-c4c6-4362-b0c6-f3149a5f299f-kube-api-access-6ls2w\") pod \"ingress-canary-wkdkt\" (UID: \"ac898250-c4c6-4362-b0c6-f3149a5f299f\") " pod="openshift-ingress-canary/ingress-canary-wkdkt"
Jul 28 19:35:22.898040 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.897977    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-certificates\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898040 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898000    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac898250-c4c6-4362-b0c6-f3149a5f299f-cert\") pod \"ingress-canary-wkdkt\" (UID: \"ac898250-c4c6-4362-b0c6-f3149a5f299f\") " pod="openshift-ingress-canary/ingress-canary-wkdkt"
Jul 28 19:35:22.898183 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898142    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-bound-sa-token\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898237 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898179    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.898237 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898208    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-trusted-ca\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898339 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898241    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce1caf2-c10b-4415-9058-7688ea24cafd-config-volume\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.898339 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898266    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khttl\" (UniqueName: \"kubernetes.io/projected/9ce1caf2-c10b-4415-9058-7688ea24cafd-kube-api-access-khttl\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.898339 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898316    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898486 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898360    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-installation-pull-secrets\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898486 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898386    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jn6c8\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-kube-api-access-jn6c8\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898486 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ce1caf2-c10b-4415-9058-7688ea24cafd-tmp-dir\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.898486 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898444    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-ca-trust-extracted\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898669 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:22.898554    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:35:22.898720 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898677    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-certificates\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.898720 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:22.898695    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:35:22.898720 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:22.898715    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db6c45f8b-r74tv: secret "image-registry-tls" not found
Jul 28 19:35:22.898860 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:22.898739    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls podName:9ce1caf2-c10b-4415-9058-7688ea24cafd nodeName:}" failed. No retries permitted until 2026-07-28 19:35:23.398719226 +0000 UTC m=+36.210281317 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls") pod "dns-default-gzdh7" (UID: "9ce1caf2-c10b-4415-9058-7688ea24cafd") : secret "dns-default-metrics-tls" not found
Jul 28 19:35:22.898860 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.898741    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9ce1caf2-c10b-4415-9058-7688ea24cafd-tmp-dir\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.898860 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:22.898781    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls podName:9c14f47a-1bfb-4c6c-94a2-46f6acfd364c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:23.398763062 +0000 UTC m=+36.210324924 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls") pod "image-registry-6db6c45f8b-r74tv" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c") : secret "image-registry-tls" not found
Jul 28 19:35:22.899179 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.899159    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/9ce1caf2-c10b-4415-9058-7688ea24cafd-config-volume\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.899288 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.899267    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-trusted-ca\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.902402 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.902279    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-image-registry-private-configuration\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.902476 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.902403    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ac898250-c4c6-4362-b0c6-f3149a5f299f-cert\") pod \"ingress-canary-wkdkt\" (UID: \"ac898250-c4c6-4362-b0c6-f3149a5f299f\") " pod="openshift-ingress-canary/ingress-canary-wkdkt"
Jul 28 19:35:22.902476 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.902304    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-installation-pull-secrets\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.909770 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.909741    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khttl\" (UniqueName: \"kubernetes.io/projected/9ce1caf2-c10b-4415-9058-7688ea24cafd-kube-api-access-khttl\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:22.910086 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.910071    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-bound-sa-token\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.910196 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.910174    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jn6c8\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-kube-api-access-jn6c8\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:22.910274 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.910259    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6ls2w\" (UniqueName: \"kubernetes.io/projected/ac898250-c4c6-4362-b0c6-f3149a5f299f-kube-api-access-6ls2w\") pod \"ingress-canary-wkdkt\" (UID: \"ac898250-c4c6-4362-b0c6-f3149a5f299f\") " pod="openshift-ingress-canary/ingress-canary-wkdkt"
Jul 28 19:35:22.974758 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:22.974727    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-wkdkt"
Jul 28 19:35:23.150658 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.150617    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-wkdkt"]
Jul 28 19:35:23.208125 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:23.208092    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac898250_c4c6_4362_b0c6_f3149a5f299f.slice/crio-ea733d7cf066f7e2cd7ac39c86f710ee91b0cc00d90e46a15bb25a113c97fc6c WatchSource:0}: Error finding container ea733d7cf066f7e2cd7ac39c86f710ee91b0cc00d90e46a15bb25a113c97fc6c: Status 404 returned error can't find the container with id ea733d7cf066f7e2cd7ac39c86f710ee91b0cc00d90e46a15bb25a113c97fc6c
Jul 28 19:35:23.401711 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.401681    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:23.401811 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.401742    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:23.401876 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:23.401834    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:35:23.401928 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:23.401877    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:35:23.401928 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:23.401890    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db6c45f8b-r74tv: secret "image-registry-tls" not found
Jul 28 19:35:23.401928 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:23.401906    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls podName:9ce1caf2-c10b-4415-9058-7688ea24cafd nodeName:}" failed. No retries permitted until 2026-07-28 19:35:24.401884971 +0000 UTC m=+37.213446828 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls") pod "dns-default-gzdh7" (UID: "9ce1caf2-c10b-4415-9058-7688ea24cafd") : secret "dns-default-metrics-tls" not found
Jul 28 19:35:23.402087 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:23.401940    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls podName:9c14f47a-1bfb-4c6c-94a2-46f6acfd364c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:24.401928567 +0000 UTC m=+37.213490556 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls") pod "image-registry-6db6c45f8b-r74tv" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c") : secret "image-registry-tls" not found
Jul 28 19:35:23.670509 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.670479    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:23.670509 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.670501    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:23.670698 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.670479    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:23.673462 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.673440    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 28 19:35:23.674122 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.674104    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vbfhc\""
Jul 28 19:35:23.674214 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.674175    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:35:23.674214 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.674206    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:35:23.674315 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.674273    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jglff\""
Jul 28 19:35:23.674494 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.674478    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 28 19:35:23.965974 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.965894    2568 generic.go:356] "Generic (PLEG): container finished" podID="ab602a3a-9515-4404-b7f9-695f78c2b49b" containerID="279237928e6ee852efe2ca0d689a4cb5a60a01404c26a7b030e36b82994c78e3" exitCode=0
Jul 28 19:35:23.966759 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.965979    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" event={"ID":"ab602a3a-9515-4404-b7f9-695f78c2b49b","Type":"ContainerDied","Data":"279237928e6ee852efe2ca0d689a4cb5a60a01404c26a7b030e36b82994c78e3"}
Jul 28 19:35:23.967051 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:23.967018    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wkdkt" event={"ID":"ac898250-c4c6-4362-b0c6-f3149a5f299f","Type":"ContainerStarted","Data":"ea733d7cf066f7e2cd7ac39c86f710ee91b0cc00d90e46a15bb25a113c97fc6c"}
Jul 28 19:35:24.410716 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:24.410683    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:24.410884 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:24.410741    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:24.410884 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:24.410863    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:35:24.411004 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:24.410941    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls podName:9ce1caf2-c10b-4415-9058-7688ea24cafd nodeName:}" failed. No retries permitted until 2026-07-28 19:35:26.410923331 +0000 UTC m=+39.222485176 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls") pod "dns-default-gzdh7" (UID: "9ce1caf2-c10b-4415-9058-7688ea24cafd") : secret "dns-default-metrics-tls" not found
Jul 28 19:35:24.411004 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:24.410865    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:35:24.411004 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:24.410988    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db6c45f8b-r74tv: secret "image-registry-tls" not found
Jul 28 19:35:24.411114 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:24.411055    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls podName:9c14f47a-1bfb-4c6c-94a2-46f6acfd364c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:26.411037606 +0000 UTC m=+39.222599450 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls") pod "image-registry-6db6c45f8b-r74tv" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c") : secret "image-registry-tls" not found
Jul 28 19:35:24.972302 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:24.972107    2568 generic.go:356] "Generic (PLEG): container finished" podID="ab602a3a-9515-4404-b7f9-695f78c2b49b" containerID="8739897cd226bc5b1f2124e1ee0628b808e91b6e076dd6ee1ee66c99b9ce2906" exitCode=0
Jul 28 19:35:24.972767 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:24.972363    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" event={"ID":"ab602a3a-9515-4404-b7f9-695f78c2b49b","Type":"ContainerDied","Data":"8739897cd226bc5b1f2124e1ee0628b808e91b6e076dd6ee1ee66c99b9ce2906"}
Jul 28 19:35:25.894229 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:25.894151    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:25.976127 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:25.976025    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-wkdkt" event={"ID":"ac898250-c4c6-4362-b0c6-f3149a5f299f","Type":"ContainerStarted","Data":"1354da7d250e12ecfd2ccf991fa88fb40cd51e1afffc73403279e77f291de780"}
Jul 28 19:35:25.979357 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:25.979309    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" event={"ID":"ab602a3a-9515-4404-b7f9-695f78c2b49b","Type":"ContainerStarted","Data":"2836282419fa2ffb1273c0934a3fa41a573e6a12e7067b16ee06413e7621fb89"}
Jul 28 19:35:25.992636 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:25.992561    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-wkdkt" podStartSLOduration=1.486635299 podStartE2EDuration="3.992541718s" totalImagesPullingTime="2.505906419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:25.992415477 +0000 UTC m=+38.803977342" watchObservedRunningTime="2026-07-28 19:35:25.992541718 +0000 UTC m=+38.804103582"
Jul 28 19:35:26.012370 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:26.012315    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:26.016455 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:26.016414    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7x4m5" podStartSLOduration=8.980627152 podStartE2EDuration="39.016395964s" totalImagesPullingTime="30.035768812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:47 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:26.015674925 +0000 UTC m=+38.827236809" watchObservedRunningTime="2026-07-28 19:35:26.016395964 +0000 UTC m=+38.827957830"
Jul 28 19:35:26.350489 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:26.350407    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:26.427124 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:26.427070    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:26.427288 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:26.427234    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:35:26.427288 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:26.427252    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db6c45f8b-r74tv: secret "image-registry-tls" not found
Jul 28 19:35:26.427288 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:26.427264    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:26.427489 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:26.427304    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls podName:9c14f47a-1bfb-4c6c-94a2-46f6acfd364c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:30.427288915 +0000 UTC m=+43.238850756 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls") pod "image-registry-6db6c45f8b-r74tv" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c") : secret "image-registry-tls" not found
Jul 28 19:35:26.427489 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:26.427400    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:35:26.427489 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:26.427470    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls podName:9ce1caf2-c10b-4415-9058-7688ea24cafd nodeName:}" failed. No retries permitted until 2026-07-28 19:35:30.42745399 +0000 UTC m=+43.239015848 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls") pod "dns-default-gzdh7" (UID: "9ce1caf2-c10b-4415-9058-7688ea24cafd") : secret "dns-default-metrics-tls" not found
Jul 28 19:35:26.450835 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:26.450812    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:30.455148 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.455111    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:30.455558 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.455191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:30.455558 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:30.455273    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:35:30.455558 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:30.455277    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:35:30.455558 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:30.455297    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db6c45f8b-r74tv: secret "image-registry-tls" not found
Jul 28 19:35:30.455558 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:30.455340    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls podName:9ce1caf2-c10b-4415-9058-7688ea24cafd nodeName:}" failed. No retries permitted until 2026-07-28 19:35:38.455309556 +0000 UTC m=+51.266871397 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls") pod "dns-default-gzdh7" (UID: "9ce1caf2-c10b-4415-9058-7688ea24cafd") : secret "dns-default-metrics-tls" not found
Jul 28 19:35:30.455558 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:30.455355    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls podName:9c14f47a-1bfb-4c6c-94a2-46f6acfd364c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:38.455348951 +0000 UTC m=+51.266910792 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls") pod "image-registry-6db6c45f8b-r74tv" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c") : secret "image-registry-tls" not found
Jul 28 19:35:30.694141 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.694106    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"]
Jul 28 19:35:30.726807 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.726741    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"]
Jul 28 19:35:30.726925 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.726871    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.729403 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.729369    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-pvgmv\""
Jul 28 19:35:30.729403 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.729369    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
Jul 28 19:35:30.729620 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.729394    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
Jul 28 19:35:30.729620 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.729475    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:35:30.729741 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.729726    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
Jul 28 19:35:30.744842 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.744819    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr"]
Jul 28 19:35:30.744944 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.744933    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.747264 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.747246    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
Jul 28 19:35:30.747475 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.747454    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 28 19:35:30.747568 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.747481    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-965px\""
Jul 28 19:35:30.747615 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.747581    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 28 19:35:30.748494 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.748471    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
Jul 28 19:35:30.758889 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.758871    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"]
Jul 28 19:35:30.758966 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.758905    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr"]
Jul 28 19:35:30.758966 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.758926    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"]
Jul 28 19:35:30.759047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.758970    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr"
Jul 28 19:35:30.761675 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.761659    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:35:30.762069 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.762048    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Jul 28 19:35:30.762069 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.762064    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-d29wq\""
Jul 28 19:35:30.794494 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.794466    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5"]
Jul 28 19:35:30.819077 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.819049    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"]
Jul 28 19:35:30.819223 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.819205    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5"
Jul 28 19:35:30.821570 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.821549    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-brlzz\""
Jul 28 19:35:30.843157 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.843134    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"]
Jul 28 19:35:30.843255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.843240    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:30.846821 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.846803    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-lbn26\""
Jul 28 19:35:30.847009 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.846995    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
Jul 28 19:35:30.847832 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.847814    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:35:30.848307 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.848291    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
Jul 28 19:35:30.857712 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.857693    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06a4de9b-a9f5-4ff3-be14-6c8d834da836-config\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.857794 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.857746    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.857794 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.857765    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.857881 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.857794    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndkdv\" (UniqueName: \"kubernetes.io/projected/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-kube-api-access-ndkdv\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.857881 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.857810    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sjpj\" (UniqueName: \"kubernetes.io/projected/06a4de9b-a9f5-4ff3-be14-6c8d834da836-kube-api-access-7sjpj\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.857881 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.857827    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06a4de9b-a9f5-4ff3-be14-6c8d834da836-serving-cert\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.857881 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.857850    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnqk8\" (UniqueName: \"kubernetes.io/projected/e49ccaaa-5113-4785-82ab-5414bea2f8f6-kube-api-access-vnqk8\") pod \"volume-data-source-validator-549c6d54cf-k96hr\" (UID: \"e49ccaaa-5113-4785-82ab-5414bea2f8f6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr"
Jul 28 19:35:30.867408 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.867389    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9zrns"]
Jul 28 19:35:30.867473 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.867459    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:30.870328 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.870297    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Jul 28 19:35:30.870600 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.870585    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-m4grt\""
Jul 28 19:35:30.870649 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.870618    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Jul 28 19:35:30.870844 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.870831    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Jul 28 19:35:30.870886 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.870858    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:35:30.891500 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.891482    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-5d9bfc7fdf-nmlwq"]
Jul 28 19:35:30.891620 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.891606    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:30.894087 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.894065    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
Jul 28 19:35:30.894188 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.894100    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
Jul 28 19:35:30.894188 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.894107    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:35:30.894188 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.894120    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
Jul 28 19:35:30.894332 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.894235    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-99nwf\""
Jul 28 19:35:30.909663 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.909642    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
Jul 28 19:35:30.911780 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.911766    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5"]
Jul 28 19:35:30.911842 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.911790    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"]
Jul 28 19:35:30.911842 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.911814    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d9bfc7fdf-nmlwq"]
Jul 28 19:35:30.911842 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.911828    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"]
Jul 28 19:35:30.911842 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.911840    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9zrns"]
Jul 28 19:35:30.912003 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.911871    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:30.912085 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.911873    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-5885fdb759-6qp7r"]
Jul 28 19:35:30.914703 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.914686    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Jul 28 19:35:30.914797 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.914703    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Jul 28 19:35:30.914965 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.914950    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Jul 28 19:35:30.915253 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.915239    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 28 19:35:30.915352 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.915331    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Jul 28 19:35:30.915405 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.915351    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 28 19:35:30.915405 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.915361    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z7j6f\""
Jul 28 19:35:30.932000 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.931982    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-6qp7r"]
Jul 28 19:35:30.932091 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.932057    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:30.934432 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.934414    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Jul 28 19:35:30.935514 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.935365    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 28 19:35:30.935514 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.935429    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Jul 28 19:35:30.935644 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.935583    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 28 19:35:30.935644 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.935628    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-st72l\""
Jul 28 19:35:30.942531 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.942504    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Jul 28 19:35:30.958619 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958597    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnqk8\" (UniqueName: \"kubernetes.io/projected/e49ccaaa-5113-4785-82ab-5414bea2f8f6-kube-api-access-vnqk8\") pod \"volume-data-source-validator-549c6d54cf-k96hr\" (UID: \"e49ccaaa-5113-4785-82ab-5414bea2f8f6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr"
Jul 28 19:35:30.958735 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958643    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:30.958735 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958665    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/192fad1c-0712-4330-b39f-a094910a4770-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:30.958815 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958795    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jjbt\" (UniqueName: \"kubernetes.io/projected/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-kube-api-access-4jjbt\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:30.958850 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958837    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ndkdv\" (UniqueName: \"kubernetes.io/projected/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-kube-api-access-ndkdv\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.958884 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958854    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2bgk\" (UniqueName: \"kubernetes.io/projected/2690519e-f882-4078-bca8-347b8809bdf9-kube-api-access-h2bgk\") pod \"network-check-source-7d88cb55d8-xdqw5\" (UID: \"2690519e-f882-4078-bca8-347b8809bdf9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5"
Jul 28 19:35:30.958916 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958906    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.958947 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958932    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sjpj\" (UniqueName: \"kubernetes.io/projected/06a4de9b-a9f5-4ff3-be14-6c8d834da836-kube-api-access-7sjpj\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.958981 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.958966    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06a4de9b-a9f5-4ff3-be14-6c8d834da836-config\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.959013 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:30.958992    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:30.959042 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.959019    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rklc\" (UniqueName: \"kubernetes.io/projected/192fad1c-0712-4330-b39f-a094910a4770-kube-api-access-6rklc\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:30.959089 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:30.959078    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls podName:0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:31.459062233 +0000 UTC m=+44.270624100 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kgnw9" (UID: "0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:30.959196 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.959175    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.959241 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.959216    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/192fad1c-0712-4330-b39f-a094910a4770-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:30.959272 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.959252    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06a4de9b-a9f5-4ff3-be14-6c8d834da836-serving-cert\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.959668 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.959647    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/06a4de9b-a9f5-4ff3-be14-6c8d834da836-config\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.959764 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.959747    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-telemetry-config\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.962448 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.962430    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/06a4de9b-a9f5-4ff3-be14-6c8d834da836-serving-cert\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:30.974215 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.974187    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnqk8\" (UniqueName: \"kubernetes.io/projected/e49ccaaa-5113-4785-82ab-5414bea2f8f6-kube-api-access-vnqk8\") pod \"volume-data-source-validator-549c6d54cf-k96hr\" (UID: \"e49ccaaa-5113-4785-82ab-5414bea2f8f6\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr"
Jul 28 19:35:30.974658 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.974642    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndkdv\" (UniqueName: \"kubernetes.io/projected/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-kube-api-access-ndkdv\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:30.974707 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:30.974686    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sjpj\" (UniqueName: \"kubernetes.io/projected/06a4de9b-a9f5-4ff3-be14-6c8d834da836-kube-api-access-7sjpj\") pod \"service-ca-operator-5d6c75f84f-d48hz\" (UID: \"06a4de9b-a9f5-4ff3-be14-6c8d834da836\") " pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:31.015369 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.015287    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:31.036155 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.036125    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"
Jul 28 19:35:31.060243 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060220    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rklc\" (UniqueName: \"kubernetes.io/projected/192fad1c-0712-4330-b39f-a094910a4770-kube-api-access-6rklc\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:31.060350 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060260    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.060350 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060280    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52f051cf-a638-429d-b7e1-3091504ad33e-config\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.060350 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060298    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/192fad1c-0712-4330-b39f-a094910a4770-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:31.060350 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060337    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-default-certificate\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.060535 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060360    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/95d94912-66ec-4945-a6ba-4c551736c24e-serving-cert\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.060535 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060417    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95d94912-66ec-4945-a6ba-4c551736c24e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.060535 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060470    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:31.060535 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060492    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/192fad1c-0712-4330-b39f-a094910a4770-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:31.060675 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060540    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jjbt\" (UniqueName: \"kubernetes.io/projected/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-kube-api-access-4jjbt\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:31.060675 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.060608    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:35:31.060675 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060652    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/95d94912-66ec-4945-a6ba-4c551736c24e-snapshots\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.060675 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.060669    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls podName:1c5336ee-6ac0-469a-9dc4-3655eee2f22c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:31.560649562 +0000 UTC m=+44.372211418 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls") pod "cluster-samples-operator-6748489656-gvc2c" (UID: "1c5336ee-6ac0-469a-9dc4-3655eee2f22c") : secret "samples-operator-tls" not found
Jul 28 19:35:31.060869 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060705    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9h4b2\" (UniqueName: \"kubernetes.io/projected/95d94912-66ec-4945-a6ba-4c551736c24e-kube-api-access-9h4b2\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.060869 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060745    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2bgk\" (UniqueName: \"kubernetes.io/projected/2690519e-f882-4078-bca8-347b8809bdf9-kube-api-access-h2bgk\") pod \"network-check-source-7d88cb55d8-xdqw5\" (UID: \"2690519e-f882-4078-bca8-347b8809bdf9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5"
Jul 28 19:35:31.060869 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060773    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52f051cf-a638-429d-b7e1-3091504ad33e-serving-cert\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.060869 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060824    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95d94912-66ec-4945-a6ba-4c551736c24e-service-ca-bundle\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.060869 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060843    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.060869 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060859    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52f051cf-a638-429d-b7e1-3091504ad33e-trusted-ca\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.061136 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060891    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-stats-auth\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.061136 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060927    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rfsf\" (UniqueName: \"kubernetes.io/projected/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-kube-api-access-5rfsf\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.061136 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.060974    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pntb\" (UniqueName: \"kubernetes.io/projected/52f051cf-a638-429d-b7e1-3091504ad33e-kube-api-access-7pntb\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.061136 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.061005    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95d94912-66ec-4945-a6ba-4c551736c24e-tmp\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.061136 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.061022    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/192fad1c-0712-4330-b39f-a094910a4770-config\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:31.062642 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.062618    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/192fad1c-0712-4330-b39f-a094910a4770-serving-cert\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:31.066690 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.066671    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr"
Jul 28 19:35:31.069858 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.069786    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rklc\" (UniqueName: \"kubernetes.io/projected/192fad1c-0712-4330-b39f-a094910a4770-kube-api-access-6rklc\") pod \"kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79\" (UID: \"192fad1c-0712-4330-b39f-a094910a4770\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:31.070168 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.070145    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jjbt\" (UniqueName: \"kubernetes.io/projected/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-kube-api-access-4jjbt\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:31.070836 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.070669    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2bgk\" (UniqueName: \"kubernetes.io/projected/2690519e-f882-4078-bca8-347b8809bdf9-kube-api-access-h2bgk\") pod \"network-check-source-7d88cb55d8-xdqw5\" (UID: \"2690519e-f882-4078-bca8-347b8809bdf9\") " pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5"
Jul 28 19:35:31.116939 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.116899    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:31.128366 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.128340    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5"
Jul 28 19:35:31.161805 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.161761    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/95d94912-66ec-4945-a6ba-4c551736c24e-snapshots\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.161973    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9h4b2\" (UniqueName: \"kubernetes.io/projected/95d94912-66ec-4945-a6ba-4c551736c24e-kube-api-access-9h4b2\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52f051cf-a638-429d-b7e1-3091504ad33e-serving-cert\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162104    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95d94912-66ec-4945-a6ba-4c551736c24e-service-ca-bundle\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162131    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162166    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52f051cf-a638-429d-b7e1-3091504ad33e-trusted-ca\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162204    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-stats-auth\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162230    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rfsf\" (UniqueName: \"kubernetes.io/projected/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-kube-api-access-5rfsf\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162253    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pntb\" (UniqueName: \"kubernetes.io/projected/52f051cf-a638-429d-b7e1-3091504ad33e-kube-api-access-7pntb\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162286    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95d94912-66ec-4945-a6ba-4c551736c24e-tmp\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162376    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52f051cf-a638-429d-b7e1-3091504ad33e-config\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162442    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-default-certificate\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162468    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/95d94912-66ec-4945-a6ba-4c551736c24e-serving-cert\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162476    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/95d94912-66ec-4945-a6ba-4c551736c24e-snapshots\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162529    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95d94912-66ec-4945-a6ba-4c551736c24e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.163221 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.162979    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95d94912-66ec-4945-a6ba-4c551736c24e-service-ca-bundle\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.164854 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.163836    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:31.663805901 +0000 UTC m=+44.475367755 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : configmap references non-existent config key: service-ca.crt
Jul 28 19:35:31.164854 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.163967    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/95d94912-66ec-4945-a6ba-4c551736c24e-trusted-ca-bundle\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.164854 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.164091    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:35:31.164854 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.164138    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:31.664123101 +0000 UTC m=+44.475684961 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : secret "router-metrics-certs-default" not found
Jul 28 19:35:31.164854 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.164746    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/52f051cf-a638-429d-b7e1-3091504ad33e-config\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.166230 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.165231    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/95d94912-66ec-4945-a6ba-4c551736c24e-tmp\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.166230 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.166152    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/52f051cf-a638-429d-b7e1-3091504ad33e-trusted-ca\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.166785 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.166716    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/52f051cf-a638-429d-b7e1-3091504ad33e-serving-cert\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.168061 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.168039    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/95d94912-66ec-4945-a6ba-4c551736c24e-serving-cert\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.168384 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.168366    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-default-certificate\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.168525 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.168505    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-stats-auth\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.173052 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.173010    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9h4b2\" (UniqueName: \"kubernetes.io/projected/95d94912-66ec-4945-a6ba-4c551736c24e-kube-api-access-9h4b2\") pod \"insights-operator-5885fdb759-6qp7r\" (UID: \"95d94912-66ec-4945-a6ba-4c551736c24e\") " pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.173729 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.173706    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pntb\" (UniqueName: \"kubernetes.io/projected/52f051cf-a638-429d-b7e1-3091504ad33e-kube-api-access-7pntb\") pod \"console-operator-584dcb6b54-9zrns\" (UID: \"52f051cf-a638-429d-b7e1-3091504ad33e\") " pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.174232 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.174192    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rfsf\" (UniqueName: \"kubernetes.io/projected/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-kube-api-access-5rfsf\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.175131 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.175052    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"
Jul 28 19:35:31.200285 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.200150    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:31.213230 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.213042    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz"]
Jul 28 19:35:31.215484 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:31.215455    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06a4de9b_a9f5_4ff3_be14_6c8d834da836.slice/crio-2125ba3e2714f24393a670955d47b61272ae1ce4ac89ccd6028c82ba53073cbc WatchSource:0}: Error finding container 2125ba3e2714f24393a670955d47b61272ae1ce4ac89ccd6028c82ba53073cbc: Status 404 returned error can't find the container with id 2125ba3e2714f24393a670955d47b61272ae1ce4ac89ccd6028c82ba53073cbc
Jul 28 19:35:31.240045 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.239965    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr"]
Jul 28 19:35:31.241521 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.241433    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-5885fdb759-6qp7r"
Jul 28 19:35:31.244805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:31.244775    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49ccaaa_5113_4785_82ab_5414bea2f8f6.slice/crio-68716ce723e047c1ef2a20002c62c54daeb62e97e8a3722dc93ddb0edac978b6 WatchSource:0}: Error finding container 68716ce723e047c1ef2a20002c62c54daeb62e97e8a3722dc93ddb0edac978b6: Status 404 returned error can't find the container with id 68716ce723e047c1ef2a20002c62c54daeb62e97e8a3722dc93ddb0edac978b6
Jul 28 19:35:31.299514 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.296826    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5"]
Jul 28 19:35:31.359641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.359612    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79"]
Jul 28 19:35:31.363504 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:31.363476    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192fad1c_0712_4330_b39f_a094910a4770.slice/crio-a8e3c664c80a1bf61577efdb5227b91289698b97c9736cab96fe9fdcd423ed1e WatchSource:0}: Error finding container a8e3c664c80a1bf61577efdb5227b91289698b97c9736cab96fe9fdcd423ed1e: Status 404 returned error can't find the container with id a8e3c664c80a1bf61577efdb5227b91289698b97c9736cab96fe9fdcd423ed1e
Jul 28 19:35:31.379625 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.379549    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-584dcb6b54-9zrns"]
Jul 28 19:35:31.382180 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:31.382142    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52f051cf_a638_429d_b7e1_3091504ad33e.slice/crio-b0974861d0ebd985402d94d877275154b69f041d41699bc5ebcdd63e216d4563 WatchSource:0}: Error finding container b0974861d0ebd985402d94d877275154b69f041d41699bc5ebcdd63e216d4563: Status 404 returned error can't find the container with id b0974861d0ebd985402d94d877275154b69f041d41699bc5ebcdd63e216d4563
Jul 28 19:35:31.408181 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.408147    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-5885fdb759-6qp7r"]
Jul 28 19:35:31.410719 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:31.410696    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95d94912_66ec_4945_a6ba_4c551736c24e.slice/crio-d08f7cdb17b8753aa292e6555d2df5065b6d3a33631b87a07caafcfaed499a99 WatchSource:0}: Error finding container d08f7cdb17b8753aa292e6555d2df5065b6d3a33631b87a07caafcfaed499a99: Status 404 returned error can't find the container with id d08f7cdb17b8753aa292e6555d2df5065b6d3a33631b87a07caafcfaed499a99
Jul 28 19:35:31.465605 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.465572    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:31.465987 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.465748    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:31.465987 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.465823    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls podName:0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:32.465802036 +0000 UTC m=+45.277363892 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kgnw9" (UID: "0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:31.566424 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.566347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:31.566575 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.566486    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:35:31.566575 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.566563    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls podName:1c5336ee-6ac0-469a-9dc4-3655eee2f22c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:32.56654066 +0000 UTC m=+45.378102508 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls") pod "cluster-samples-operator-6748489656-gvc2c" (UID: "1c5336ee-6ac0-469a-9dc4-3655eee2f22c") : secret "samples-operator-tls" not found
Jul 28 19:35:31.666959 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.666928    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.667116 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.666984    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:31.667116 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.667086    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:35:31.667116 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.667099    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:32.667078759 +0000 UTC m=+45.478640602 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : configmap references non-existent config key: service-ca.crt
Jul 28 19:35:31.667233 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:31.667125    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:32.667113245 +0000 UTC m=+45.478675087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : secret "router-metrics-certs-default" not found
Jul 28 19:35:31.998210 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:31.998139    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-6qp7r" event={"ID":"95d94912-66ec-4945-a6ba-4c551736c24e","Type":"ContainerStarted","Data":"d08f7cdb17b8753aa292e6555d2df5065b6d3a33631b87a07caafcfaed499a99"}
Jul 28 19:35:32.002391 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.002293    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns" event={"ID":"52f051cf-a638-429d-b7e1-3091504ad33e","Type":"ContainerStarted","Data":"b0974861d0ebd985402d94d877275154b69f041d41699bc5ebcdd63e216d4563"}
Jul 28 19:35:32.003785 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.003723    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz" event={"ID":"06a4de9b-a9f5-4ff3-be14-6c8d834da836","Type":"ContainerStarted","Data":"2125ba3e2714f24393a670955d47b61272ae1ce4ac89ccd6028c82ba53073cbc"}
Jul 28 19:35:32.005895 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.005845    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5" event={"ID":"2690519e-f882-4078-bca8-347b8809bdf9","Type":"ContainerStarted","Data":"9e39361cd819f16997fd72601919a59290c005836ae2f67b0dd68c7a4782b26a"}
Jul 28 19:35:32.007149 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.007093    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79" event={"ID":"192fad1c-0712-4330-b39f-a094910a4770","Type":"ContainerStarted","Data":"a8e3c664c80a1bf61577efdb5227b91289698b97c9736cab96fe9fdcd423ed1e"}
Jul 28 19:35:32.008729 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.008702    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr" event={"ID":"e49ccaaa-5113-4785-82ab-5414bea2f8f6","Type":"ContainerStarted","Data":"68716ce723e047c1ef2a20002c62c54daeb62e97e8a3722dc93ddb0edac978b6"}
Jul 28 19:35:32.241599 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.241560    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:32.310463 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.310358    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:32.477650 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.476987    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:32.477650 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:32.477230    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:32.477650 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:32.477296    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls podName:0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:34.477276636 +0000 UTC m=+47.288838500 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kgnw9" (UID: "0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:32.579187 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.578376    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:32.579187 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:32.578620    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:35:32.579187 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:32.578687    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls podName:1c5336ee-6ac0-469a-9dc4-3655eee2f22c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:34.578666849 +0000 UTC m=+47.390228709 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls") pod "cluster-samples-operator-6748489656-gvc2c" (UID: "1c5336ee-6ac0-469a-9dc4-3655eee2f22c") : secret "samples-operator-tls" not found
Jul 28 19:35:32.680523 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.679730    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:32.680523 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:32.679823    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:32.680523 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:32.679946    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:35:32.680523 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:32.680008    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:34.67999017 +0000 UTC m=+47.491552028 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : secret "router-metrics-certs-default" not found
Jul 28 19:35:32.680523 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:32.680434    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:34.680416997 +0000 UTC m=+47.491978852 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : configmap references non-existent config key: service-ca.crt
Jul 28 19:35:34.016069 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:34.016035    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr" event={"ID":"e49ccaaa-5113-4785-82ab-5414bea2f8f6","Type":"ContainerStarted","Data":"bdb0199f3af4c1d21e895905a2f10817eb7bce7cd543f3cfdb011c65fb9b3726"}
Jul 28 19:35:34.108696 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:34.108659    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:34.498021 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:34.497984    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:34.498207 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:34.498160    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:34.498275 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:34.498244    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls podName:0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:38.498222781 +0000 UTC m=+51.309784639 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kgnw9" (UID: "0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:34.598721 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:34.598682    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:34.598877 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:34.598837    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:35:34.598924 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:34.598905    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls podName:1c5336ee-6ac0-469a-9dc4-3655eee2f22c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:38.598889892 +0000 UTC m=+51.410451750 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls") pod "cluster-samples-operator-6748489656-gvc2c" (UID: "1c5336ee-6ac0-469a-9dc4-3655eee2f22c") : secret "samples-operator-tls" not found
Jul 28 19:35:34.700001 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:34.699964    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:34.700166 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:34.700041    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:34.700166 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:34.700083    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:38.700066601 +0000 UTC m=+51.511628442 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : configmap references non-existent config key: service-ca.crt
Jul 28 19:35:34.700166 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:34.700152    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:35:34.700315 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:34.700209    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:38.700192648 +0000 UTC m=+51.511754505 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : secret "router-metrics-certs-default" not found
Jul 28 19:35:35.130725 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.130670    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-549c6d54cf-k96hr" podStartSLOduration=3.210458307 podStartE2EDuration="5.130650507s" totalImagesPullingTime="1.9201922s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:34.030633327 +0000 UTC m=+46.842195190" watchObservedRunningTime="2026-07-28 19:35:35.130650507 +0000 UTC m=+47.942212370"
Jul 28 19:35:35.131115 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.130867    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"]
Jul 28 19:35:35.159551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.159528    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"]
Jul 28 19:35:35.159715 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.159627    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:35.162383 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.162354    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Jul 28 19:35:35.162383 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.162373    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Jul 28 19:35:35.162593 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.162491    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-55bmj\""
Jul 28 19:35:35.205157 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.205116    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:35.205157 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.205160    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:35.259316 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.259267    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:35.306371 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.306338    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:35.306371 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.306374    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:35.306586 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:35.306501    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:35:35.306586 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:35.306566    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert podName:e5087f7a-f48f-4dc9-a7ac-d1a18643367b nodeName:}" failed. No retries permitted until 2026-07-28 19:35:35.806547345 +0000 UTC m=+48.618109185 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lw2hv" (UID: "e5087f7a-f48f-4dc9-a7ac-d1a18643367b") : secret "networking-console-plugin-cert" not found
Jul 28 19:35:35.307251 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.307230    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-nginx-conf\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:35.359909 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.359872    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:35.811159 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:35.811126    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:35.811342 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:35.811275    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:35:35.811389 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:35.811362    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert podName:e5087f7a-f48f-4dc9-a7ac-d1a18643367b nodeName:}" failed. No retries permitted until 2026-07-28 19:35:36.811346147 +0000 UTC m=+49.622907989 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lw2hv" (UID: "e5087f7a-f48f-4dc9-a7ac-d1a18643367b") : secret "networking-console-plugin-cert" not found
Jul 28 19:35:36.819341 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:36.819295    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:36.819720 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:36.819439    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:35:36.819720 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:36.819503    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert podName:e5087f7a-f48f-4dc9-a7ac-d1a18643367b nodeName:}" failed. No retries permitted until 2026-07-28 19:35:38.819488974 +0000 UTC m=+51.631050820 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lw2hv" (UID: "e5087f7a-f48f-4dc9-a7ac-d1a18643367b") : secret "networking-console-plugin-cert" not found
Jul 28 19:35:38.030354 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.030017    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5" event={"ID":"2690519e-f882-4078-bca8-347b8809bdf9","Type":"ContainerStarted","Data":"050dc63e8e6bc74d680e839dab429fc02db19172dd9ea46bbac5848d9c0c6f35"}
Jul 28 19:35:38.031633 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.031607    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79" event={"ID":"192fad1c-0712-4330-b39f-a094910a4770","Type":"ContainerStarted","Data":"cc8cb489242a0120ace99a8870dc9547744bd0be9ec4cc38030ab21153bc71dd"}
Jul 28 19:35:38.033696 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.033664    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-6qp7r" event={"ID":"95d94912-66ec-4945-a6ba-4c551736c24e","Type":"ContainerStarted","Data":"e71dc22db4e5b5acf3bce9523929382d6188653ba8d5a2992110e7e567c46ce1"}
Jul 28 19:35:38.035850 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.035829    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/0.log"
Jul 28 19:35:38.035949 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.035870    2568 generic.go:356] "Generic (PLEG): container finished" podID="52f051cf-a638-429d-b7e1-3091504ad33e" containerID="069c7439e6bcdb5fe3a7b60f18c0037b5bde3da02dd51c44f541ba11128fa4a8" exitCode=255
Jul 28 19:35:38.035949 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.035928    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns" event={"ID":"52f051cf-a638-429d-b7e1-3091504ad33e","Type":"ContainerDied","Data":"069c7439e6bcdb5fe3a7b60f18c0037b5bde3da02dd51c44f541ba11128fa4a8"}
Jul 28 19:35:38.036144 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.036130    2568 scope.go:117] "RemoveContainer" containerID="069c7439e6bcdb5fe3a7b60f18c0037b5bde3da02dd51c44f541ba11128fa4a8"
Jul 28 19:35:38.038715 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.038685    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz" event={"ID":"06a4de9b-a9f5-4ff3-be14-6c8d834da836","Type":"ContainerStarted","Data":"e8cb83f4a7c2a28aeacaf10ee829cf4f0a716d8c01e59a9f643cd243a614e63c"}
Jul 28 19:35:38.046523 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.046475    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7d88cb55d8-xdqw5" podStartSLOduration=1.5555421809999999 podStartE2EDuration="8.046457792s" totalImagesPullingTime="6.490915611s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:38.044725522 +0000 UTC m=+50.856287386" watchObservedRunningTime="2026-07-28 19:35:38.046457792 +0000 UTC m=+50.858019656"
Jul 28 19:35:38.065996 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.065946    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-5885fdb759-6qp7r" podStartSLOduration=1.681495784 podStartE2EDuration="8.065924924s" totalImagesPullingTime="6.38442914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:38.063973804 +0000 UTC m=+50.875535668" watchObservedRunningTime="2026-07-28 19:35:38.065924924 +0000 UTC m=+50.877486788"
Jul 28 19:35:38.085959 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.085905    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79" podStartSLOduration=1.660043007 podStartE2EDuration="8.08588225s" totalImagesPullingTime="6.425839243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:38.084660318 +0000 UTC m=+50.896222210" watchObservedRunningTime="2026-07-28 19:35:38.08588225 +0000 UTC m=+50.897444092"
Jul 28 19:35:38.090289 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.090255    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:38.124019 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.123867    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz" podStartSLOduration=1.549721118 podStartE2EDuration="8.123843158s" totalImagesPullingTime="6.57412204s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:38.102402732 +0000 UTC m=+50.913964613" watchObservedRunningTime="2026-07-28 19:35:38.123843158 +0000 UTC m=+50.935405022"
Jul 28 19:35:38.228287 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.228250    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:38.328297 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.328262    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:38.534974 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.534926    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:38.535169 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.535032    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:38.535169 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.535101    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:38.535169 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.535130    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:38.535316 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.535176    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls podName:0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:46.535155898 +0000 UTC m=+59.346717765 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kgnw9" (UID: "0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:38.535316 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.535244    2568 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 28 19:35:38.535316 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.535293    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls podName:9ce1caf2-c10b-4415-9058-7688ea24cafd nodeName:}" failed. No retries permitted until 2026-07-28 19:35:54.535278174 +0000 UTC m=+67.346840018 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls") pod "dns-default-gzdh7" (UID: "9ce1caf2-c10b-4415-9058-7688ea24cafd") : secret "dns-default-metrics-tls" not found
Jul 28 19:35:38.535509 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.535381    2568 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Jul 28 19:35:38.535509 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.535392    2568 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6db6c45f8b-r74tv: secret "image-registry-tls" not found
Jul 28 19:35:38.535509 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.535426    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls podName:9c14f47a-1bfb-4c6c-94a2-46f6acfd364c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:54.535415856 +0000 UTC m=+67.346977704 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls") pod "image-registry-6db6c45f8b-r74tv" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c") : secret "image-registry-tls" not found
Jul 28 19:35:38.636091 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.636047    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:38.636276 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.636207    2568 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
Jul 28 19:35:38.636276 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.636268    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls podName:1c5336ee-6ac0-469a-9dc4-3655eee2f22c nodeName:}" failed. No retries permitted until 2026-07-28 19:35:46.636252231 +0000 UTC m=+59.447814077 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls") pod "cluster-samples-operator-6748489656-gvc2c" (UID: "1c5336ee-6ac0-469a-9dc4-3655eee2f22c") : secret "samples-operator-tls" not found
Jul 28 19:35:38.736716 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.736674    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:38.736909 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.736825    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:38.736909 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.736854    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:35:38.737013 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.736933    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:46.736913569 +0000 UTC m=+59.548475416 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : secret "router-metrics-certs-default" not found
Jul 28 19:35:38.737013 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.736951    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:46.736942351 +0000 UTC m=+59.548504195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : configmap references non-existent config key: service-ca.crt
Jul 28 19:35:38.837256 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.837220    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:38.837580 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.837551    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:35:38.837713 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:38.837639    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert podName:e5087f7a-f48f-4dc9-a7ac-d1a18643367b nodeName:}" failed. No retries permitted until 2026-07-28 19:35:42.837616712 +0000 UTC m=+55.649178556 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lw2hv" (UID: "e5087f7a-f48f-4dc9-a7ac-d1a18643367b") : secret "networking-console-plugin-cert" not found
Jul 28 19:35:38.907682 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.907641    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mqc6s"]
Jul 28 19:35:38.915405 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.915317    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:38.925672 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.925628    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mqc6s"]
Jul 28 19:35:38.925847 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.925760    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:38.929136 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.929000    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rtz66\""
Jul 28 19:35:38.929301 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.929216    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 28 19:35:38.929301 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:38.929225    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 28 19:35:39.024176 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.024124    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.038612 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.038581    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n259n\" (UniqueName: \"kubernetes.io/projected/a5224743-992f-4d1c-86fe-750213fb90f4-kube-api-access-n259n\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.039070 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.038649    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.039070 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.038763    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.039070 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.038803    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5224743-992f-4d1c-86fe-750213fb90f4-crio-socket\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.039070 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.038901    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5224743-992f-4d1c-86fe-750213fb90f4-data-volume\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.044151 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.044116    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:35:39.044661 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.044644    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/0.log"
Jul 28 19:35:39.044755 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.044693    2568 generic.go:356] "Generic (PLEG): container finished" podID="52f051cf-a638-429d-b7e1-3091504ad33e" containerID="ba4009f402cf6a7da7230e201de1b0baee72870d569c719b2e189432c3bbbdbb" exitCode=255
Jul 28 19:35:39.044824 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.044753    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns" event={"ID":"52f051cf-a638-429d-b7e1-3091504ad33e","Type":"ContainerDied","Data":"ba4009f402cf6a7da7230e201de1b0baee72870d569c719b2e189432c3bbbdbb"}
Jul 28 19:35:39.044879 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.044847    2568 scope.go:117] "RemoveContainer" containerID="069c7439e6bcdb5fe3a7b60f18c0037b5bde3da02dd51c44f541ba11128fa4a8"
Jul 28 19:35:39.045118 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.045095    2568 scope.go:117] "RemoveContainer" containerID="ba4009f402cf6a7da7230e201de1b0baee72870d569c719b2e189432c3bbbdbb"
Jul 28 19:35:39.045451 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.045402    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9zrns_openshift-console-operator(52f051cf-a638-429d-b7e1-3091504ad33e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns" podUID="52f051cf-a638-429d-b7e1-3091504ad33e"
Jul 28 19:35:39.092619 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.092531    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.139774 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.139734    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.139967 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.139817    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.139967 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.139850    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5224743-992f-4d1c-86fe-750213fb90f4-crio-socket\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.139967 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.139907    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:39.139967 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.139944    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:35:39.140173 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.139994    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:39.639974145 +0000 UTC m=+52.451535992 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:39.140173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.140040    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5224743-992f-4d1c-86fe-750213fb90f4-data-volume\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.140173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.140084    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a5224743-992f-4d1c-86fe-750213fb90f4-crio-socket\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.140357 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.140206    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:39.640195883 +0000 UTC m=+52.451757736 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : configmap "kube-rbac-proxy" not found
Jul 28 19:35:39.140357 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.140249    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n259n\" (UniqueName: \"kubernetes.io/projected/a5224743-992f-4d1c-86fe-750213fb90f4-kube-api-access-n259n\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.140793 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.140750    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a5224743-992f-4d1c-86fe-750213fb90f4-data-volume\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.146601 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.146574    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.148834 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.148809    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n259n\" (UniqueName: \"kubernetes.io/projected/a5224743-992f-4d1c-86fe-750213fb90f4-kube-api-access-n259n\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.346512 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.346421    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.448189 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.448103    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.548665 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.548630    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.644723 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.644686    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.644898 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.644745    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:39.644898 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.644840    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:39.644898 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.644876    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:35:39.645020 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.644916    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:40.644899569 +0000 UTC m=+53.456461410 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:39.645020 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:39.644936    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:40.644924014 +0000 UTC m=+53.456485891 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : configmap "kube-rbac-proxy" not found
Jul 28 19:35:39.656854 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.656823    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.709591 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.708369    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw"]
Jul 28 19:35:39.743033 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.743005    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw"]
Jul 28 19:35:39.743175 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.743126    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw"
Jul 28 19:35:39.745682 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.745652    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 28 19:35:39.745682 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.745660    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 28 19:35:39.746989 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.746961    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-485jz\""
Jul 28 19:35:39.751046 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.751023    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.846029 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.845993    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26fv\" (UniqueName: \"kubernetes.io/projected/2bd54e8e-d9e9-469d-bb92-8c465c918ec5-kube-api-access-r26fv\") pod \"migrator-6865c9967-cscvw\" (UID: \"2bd54e8e-d9e9-469d-bb92-8c465c918ec5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw"
Jul 28 19:35:39.853644 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.853604    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:39.910582 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.910508    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jfqbz_605d46ac-45a6-48a9-a859-11a690c22efc/dns-node-resolver/0.log"
Jul 28 19:35:39.947220 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.947191    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r26fv\" (UniqueName: \"kubernetes.io/projected/2bd54e8e-d9e9-469d-bb92-8c465c918ec5-kube-api-access-r26fv\") pod \"migrator-6865c9967-cscvw\" (UID: \"2bd54e8e-d9e9-469d-bb92-8c465c918ec5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw"
Jul 28 19:35:39.954776 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.954755    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26fv\" (UniqueName: \"kubernetes.io/projected/2bd54e8e-d9e9-469d-bb92-8c465c918ec5-kube-api-access-r26fv\") pod \"migrator-6865c9967-cscvw\" (UID: \"2bd54e8e-d9e9-469d-bb92-8c465c918ec5\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw"
Jul 28 19:35:39.961577 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:39.961553    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:40.049832 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.049803    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:35:40.050312 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.050193    2568 scope.go:117] "RemoveContainer" containerID="ba4009f402cf6a7da7230e201de1b0baee72870d569c719b2e189432c3bbbdbb"
Jul 28 19:35:40.050439 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:40.050385    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9zrns_openshift-console-operator(52f051cf-a638-429d-b7e1-3091504ad33e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns" podUID="52f051cf-a638-429d-b7e1-3091504ad33e"
Jul 28 19:35:40.053265 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.053247    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw"
Jul 28 19:35:40.165241 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.165169    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:40.196431 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.196401    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw"]
Jul 28 19:35:40.199520 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:40.199493    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bd54e8e_d9e9_469d_bb92_8c465c918ec5.slice/crio-98abc0c551a9a8d9b385787c136ac8684318fd90ccb21c76c1a8ade2bf324dff WatchSource:0}: Error finding container 98abc0c551a9a8d9b385787c136ac8684318fd90ccb21c76c1a8ade2bf324dff: Status 404 returned error can't find the container with id 98abc0c551a9a8d9b385787c136ac8684318fd90ccb21c76c1a8ade2bf324dff
Jul 28 19:35:40.269529 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.269488    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:40.323553 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.323522    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:40.566675 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.566586    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:40.654314 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.654274    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:40.654527 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.654345    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:40.654527 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:40.654430    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:35:40.654527 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:40.654441    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:40.654527 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:40.654485    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:42.654470825 +0000 UTC m=+55.466032666 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : configmap "kube-rbac-proxy" not found
Jul 28 19:35:40.654527 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:40.654499    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:42.654493171 +0000 UTC m=+55.466055012 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:40.666333 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.666293    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:40.909762 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:40.909733    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mmn7j_40db1398-97f2-4b10-b6b0-debf5bdb6c5d/node-ca/0.log"
Jul 28 19:35:41.054401 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.054357    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw" event={"ID":"2bd54e8e-d9e9-469d-bb92-8c465c918ec5","Type":"ContainerStarted","Data":"98abc0c551a9a8d9b385787c136ac8684318fd90ccb21c76c1a8ade2bf324dff"}
Jul 28 19:35:41.201385 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.201294    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:41.201385 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.201374    2568 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:41.201773 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.201759    2568 scope.go:117] "RemoveContainer" containerID="ba4009f402cf6a7da7230e201de1b0baee72870d569c719b2e189432c3bbbdbb"
Jul 28 19:35:41.201948 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:41.201928    2568 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-584dcb6b54-9zrns_openshift-console-operator(52f051cf-a638-429d-b7e1-3091504ad33e)\"" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns" podUID="52f051cf-a638-429d-b7e1-3091504ad33e"
Jul 28 19:35:41.710019 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.709989    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wkdkt_ac898250-c4c6-4362-b0c6-f3149a5f299f/serve-healthcheck-canary/0.log"
Jul 28 19:35:41.838889 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.838854    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-98l9b"]
Jul 28 19:35:41.851997 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.851971    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-98l9b"]
Jul 28 19:35:41.852148 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.852081    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:41.854841 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.854817    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Jul 28 19:35:41.855747 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.855701    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Jul 28 19:35:41.855747 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.855731    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7hd9d\""
Jul 28 19:35:41.855936 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.855792    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Jul 28 19:35:41.855936 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.855792    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Jul 28 19:35:41.888813 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.888789    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:41.947074 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.947048    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:41.968207 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.968155    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a492671c-1b96-4ea7-b156-0d9b48c99b89-signing-key\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:41.968306 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.968222    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a492671c-1b96-4ea7-b156-0d9b48c99b89-signing-cabundle\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:41.968306 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:41.968241    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsj4b\" (UniqueName: \"kubernetes.io/projected/a492671c-1b96-4ea7-b156-0d9b48c99b89-kube-api-access-bsj4b\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:42.069574 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.069544    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a492671c-1b96-4ea7-b156-0d9b48c99b89-signing-key\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:42.069917 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.069616    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a492671c-1b96-4ea7-b156-0d9b48c99b89-signing-cabundle\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:42.069917 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.069639    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsj4b\" (UniqueName: \"kubernetes.io/projected/a492671c-1b96-4ea7-b156-0d9b48c99b89-kube-api-access-bsj4b\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:42.070350 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.070312    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/a492671c-1b96-4ea7-b156-0d9b48c99b89-signing-cabundle\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:42.072104 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.072082    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/a492671c-1b96-4ea7-b156-0d9b48c99b89-signing-key\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:42.077869 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.077847    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsj4b\" (UniqueName: \"kubernetes.io/projected/a492671c-1b96-4ea7-b156-0d9b48c99b89-kube-api-access-bsj4b\") pod \"service-ca-658d7b7c78-98l9b\" (UID: \"a492671c-1b96-4ea7-b156-0d9b48c99b89\") " pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:42.162091 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.162069    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-98l9b"
Jul 28 19:35:42.300442 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.300413    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-98l9b"]
Jul 28 19:35:42.302429 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:42.302389    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda492671c_1b96_4ea7_b156_0d9b48c99b89.slice/crio-011af441053491662950d12455125a96cd9d95ab505ee8a333c7ffdc76fc4017 WatchSource:0}: Error finding container 011af441053491662950d12455125a96cd9d95ab505ee8a333c7ffdc76fc4017: Status 404 returned error can't find the container with id 011af441053491662950d12455125a96cd9d95ab505ee8a333c7ffdc76fc4017
Jul 28 19:35:42.674096 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.674061    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:42.674292 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.674108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:42.674292 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:42.674177    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:35:42.674292 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:42.674198    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:42.674292 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:42.674223    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:46.674210792 +0000 UTC m=+59.485772632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : configmap "kube-rbac-proxy" not found
Jul 28 19:35:42.674292 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:42.674249    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:46.674236792 +0000 UTC m=+59.485798632 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:42.734722 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.734688    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:42.875895 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:42.875860    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:42.876177 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:42.876010    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:35:42.876177 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:42.876073    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert podName:e5087f7a-f48f-4dc9-a7ac-d1a18643367b nodeName:}" failed. No retries permitted until 2026-07-28 19:35:50.876058577 +0000 UTC m=+63.687620436 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lw2hv" (UID: "e5087f7a-f48f-4dc9-a7ac-d1a18643367b") : secret "networking-console-plugin-cert" not found
Jul 28 19:35:43.063247 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:43.063161    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw" event={"ID":"2bd54e8e-d9e9-469d-bb92-8c465c918ec5","Type":"ContainerStarted","Data":"1123e684a55880114d32ccdda49f2f0015b1cb13340563ff70908348b07b34ce"}
Jul 28 19:35:43.063247 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:43.063223    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw" event={"ID":"2bd54e8e-d9e9-469d-bb92-8c465c918ec5","Type":"ContainerStarted","Data":"33d1e33e71f3719d84fad9c27cd1c9960346968826bd5618e6cdda87d0beebb5"}
Jul 28 19:35:43.064660 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:43.064638    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-98l9b" event={"ID":"a492671c-1b96-4ea7-b156-0d9b48c99b89","Type":"ContainerStarted","Data":"4ac7c5bb5f35d3294560e0cecfe2ad63e8c7bacc90e3bf5e3fce0d19e6d4a958"}
Jul 28 19:35:43.064764 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:43.064675    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-98l9b" event={"ID":"a492671c-1b96-4ea7-b156-0d9b48c99b89","Type":"ContainerStarted","Data":"011af441053491662950d12455125a96cd9d95ab505ee8a333c7ffdc76fc4017"}
Jul 28 19:35:43.082687 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:43.082639    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-cscvw" podStartSLOduration=2.166790216 podStartE2EDuration="4.082625529s" totalImagesPullingTime="1.915835313s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:43.082190763 +0000 UTC m=+55.893752627" watchObservedRunningTime="2026-07-28 19:35:43.082625529 +0000 UTC m=+55.894187391"
Jul 28 19:35:43.753971 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:43.753937    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:43.882641 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:43.882601    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:43.980902 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:43.980872    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:44.163002 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:44.162973    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:44.263992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:44.263960    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:44.470791 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:44.470685    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:45.658831 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:45.658799    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:45.852820 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:45.852766    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:46.065736 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.065649    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:46.256979 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.256942    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:46.460094 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.460047    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:46.610560 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.610511    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:35:46.610740 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.610630    2568 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:46.610740 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.610695    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls podName:0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:02.61068057 +0000 UTC m=+75.422242410 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-548b489cb8-kgnw9" (UID: "0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3") : secret "cluster-monitoring-operator-tls" not found
Jul 28 19:35:46.711405 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.711291    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:46.711858 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.711423    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:46.711858 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.711460    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:46.711858 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.711521    2568 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 28 19:35:46.711858 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.711557    2568 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:46.711858 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.711570    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:54.711557616 +0000 UTC m=+67.523119458 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : configmap "kube-rbac-proxy" not found
Jul 28 19:35:46.711858 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.711630    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls podName:a5224743-992f-4d1c-86fe-750213fb90f4 nodeName:}" failed. No retries permitted until 2026-07-28 19:35:54.711611282 +0000 UTC m=+67.523173144 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mqc6s" (UID: "a5224743-992f-4d1c-86fe-750213fb90f4") : secret "insights-runtime-extractor-tls" not found
Jul 28 19:35:46.713635 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.713616    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/1c5336ee-6ac0-469a-9dc4-3655eee2f22c-samples-operator-tls\") pod \"cluster-samples-operator-6748489656-gvc2c\" (UID: \"1c5336ee-6ac0-469a-9dc4-3655eee2f22c\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:46.751296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.751270    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"
Jul 28 19:35:46.812429 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.812234    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:46.812429 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.812391    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:35:46.812633 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.812585    2568 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Jul 28 19:35:46.812690 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.812654    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:02.812633333 +0000 UTC m=+75.624195191 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : secret "router-metrics-certs-default" not found
Jul 28 19:35:46.813419 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:46.813127    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle podName:e9190e9f-9c61-4c82-aa8a-b14cc3e1c348 nodeName:}" failed. No retries permitted until 2026-07-28 19:36:02.813109025 +0000 UTC m=+75.624670882 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle") pod "router-default-5d9bfc7fdf-nmlwq" (UID: "e9190e9f-9c61-4c82-aa8a-b14cc3e1c348") : configmap references non-existent config key: service-ca.crt
Jul 28 19:35:46.875699 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.875644    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-98l9b" podStartSLOduration=5.875626795 podStartE2EDuration="5.875626795s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:43.099158391 +0000 UTC m=+55.910720256" watchObservedRunningTime="2026-07-28 19:35:46.875626795 +0000 UTC m=+59.687188658"
Jul 28 19:35:46.875936 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:46.875921    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c"]
Jul 28 19:35:47.077998 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:47.077918    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c" event={"ID":"1c5336ee-6ac0-469a-9dc4-3655eee2f22c","Type":"ContainerStarted","Data":"2dae2a924ea8e35f17fad33576d4c1cd784219f679d58bd455298f400dc1a9c1"}
Jul 28 19:35:47.407564 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:47.407532    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:47.515391 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:47.515352    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:47.662856 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:47.662745    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:47.790414 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:47.790380    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:47.980237 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:47.980167    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-8wcxl"
Jul 28 19:35:48.059165 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:48.059127    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:48.258338 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:48.258230    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:48.318109 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:48.318073    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:48.462207 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:48.462169    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:49.467014 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:49.466940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:49.861240 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:49.861139    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:49.974968 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:49.974928    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:50.072637 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:50.072580    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:50.090828 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:50.090799    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c" event={"ID":"1c5336ee-6ac0-469a-9dc4-3655eee2f22c","Type":"ContainerStarted","Data":"79daff22d63a3c40cf155b9937fd1e0032f54eedc0aa214ba6b153dd9c0204d4"}
Jul 28 19:35:50.090959 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:50.090872    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c" event={"ID":"1c5336ee-6ac0-469a-9dc4-3655eee2f22c","Type":"ContainerStarted","Data":"76fcbc4d665fb77bbe4c2b5a2ccc7457abe718a50b41cf3f39d645a0fb2232e7"}
Jul 28 19:35:50.108380 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:50.108312    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6748489656-gvc2c" podStartSLOduration=17.831841062 podStartE2EDuration="20.108296424s" totalImagesPullingTime="2.276455362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:50.106721569 +0000 UTC m=+62.918283432" watchObservedRunningTime="2026-07-28 19:35:50.108296424 +0000 UTC m=+62.919858290"
Jul 28 19:35:50.254287 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:50.254249    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:50.457997 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:50.457956    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:50.952202 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:50.952168    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:35:50.952600 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:50.952316    2568 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Jul 28 19:35:50.952600 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:35:50.952401    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert podName:e5087f7a-f48f-4dc9-a7ac-d1a18643367b nodeName:}" failed. No retries permitted until 2026-07-28 19:36:06.952386226 +0000 UTC m=+79.763948070 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert") pod "networking-console-plugin-74c9f884b7-lw2hv" (UID: "e5087f7a-f48f-4dc9-a7ac-d1a18643367b") : secret "networking-console-plugin-cert" not found
Jul 28 19:35:51.657173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:51.657141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:51.859124 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:51.859083    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:52.058503 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.058411    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:52.262726 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.262690    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:52.456957 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.456920    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:52.466961 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.466932    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:52.467081 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.466998    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:52.469825 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.469806    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 28 19:35:52.469896 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.469833    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 28 19:35:52.480023 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.479993    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/967eb3cb-a574-4af5-bba1-78655a105b62-metrics-certs\") pod \"network-metrics-daemon-hs6df\" (UID: \"967eb3cb-a574-4af5-bba1-78655a105b62\") " pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:52.480107 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.480072    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/66ebf5f5-ab7c-4cb7-9857-b376b202edae-original-pull-secret\") pod \"global-pull-secret-syncer-6w927\" (UID: \"66ebf5f5-ab7c-4cb7-9857-b376b202edae\") " pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:52.482881 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.482864    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-vbfhc\""
Jul 28 19:35:52.491927 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.491909    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-hs6df"
Jul 28 19:35:52.494158 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.494141    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6w927"
Jul 28 19:35:52.574388 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.568332    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:52.587296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.581019    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjfpr\" (UniqueName: \"kubernetes.io/projected/a2f635a2-a6f0-4f62-962f-2d511a939e28-kube-api-access-hjfpr\") pod \"network-check-target-rh26c\" (UID: \"a2f635a2-a6f0-4f62-962f-2d511a939e28\") " pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:52.644949 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.644926    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-hs6df"]
Jul 28 19:35:52.647900 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:52.647874    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod967eb3cb_a574_4af5_bba1_78655a105b62.slice/crio-2b54e4cd5804212b85d5ec5ac1cafa7390b5a3882573e9d0076c183d68d1c463 WatchSource:0}: Error finding container 2b54e4cd5804212b85d5ec5ac1cafa7390b5a3882573e9d0076c183d68d1c463: Status 404 returned error can't find the container with id 2b54e4cd5804212b85d5ec5ac1cafa7390b5a3882573e9d0076c183d68d1c463
Jul 28 19:35:52.668270 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.668251    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6w927"]
Jul 28 19:35:52.670728 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:52.670705    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66ebf5f5_ab7c_4cb7_9857_b376b202edae.slice/crio-ddcbcb5e2ba3b36e1c1a5ef30c1e579863e22fcc9ccb32c0eb3899143a259577 WatchSource:0}: Error finding container ddcbcb5e2ba3b36e1c1a5ef30c1e579863e22fcc9ccb32c0eb3899143a259577: Status 404 returned error can't find the container with id ddcbcb5e2ba3b36e1c1a5ef30c1e579863e22fcc9ccb32c0eb3899143a259577
Jul 28 19:35:52.790599 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.790522    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jglff\""
Jul 28 19:35:52.798405 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.798387    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:52.918460 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:52.918434    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-rh26c"]
Jul 28 19:35:52.920691 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:52.920666    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2f635a2_a6f0_4f62_962f_2d511a939e28.slice/crio-3592b4d3ad679a38808b14653e8190ac33599bbf6a05ee295ea0966a7fd4c55c WatchSource:0}: Error finding container 3592b4d3ad679a38808b14653e8190ac33599bbf6a05ee295ea0966a7fd4c55c: Status 404 returned error can't find the container with id 3592b4d3ad679a38808b14653e8190ac33599bbf6a05ee295ea0966a7fd4c55c
Jul 28 19:35:53.101756 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.101671    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hs6df" event={"ID":"967eb3cb-a574-4af5-bba1-78655a105b62","Type":"ContainerStarted","Data":"2b54e4cd5804212b85d5ec5ac1cafa7390b5a3882573e9d0076c183d68d1c463"}
Jul 28 19:35:53.102810 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.102779    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6w927" event={"ID":"66ebf5f5-ab7c-4cb7-9857-b376b202edae","Type":"ContainerStarted","Data":"ddcbcb5e2ba3b36e1c1a5ef30c1e579863e22fcc9ccb32c0eb3899143a259577"}
Jul 28 19:35:53.104095 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.104074    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rh26c" event={"ID":"a2f635a2-a6f0-4f62-962f-2d511a939e28","Type":"ContainerStarted","Data":"00e9ef947c501132cfc3f5d57d670e08563c218c08e6293b2162bf23e2d6540c"}
Jul 28 19:35:53.104196 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.104111    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-rh26c" event={"ID":"a2f635a2-a6f0-4f62-962f-2d511a939e28","Type":"ContainerStarted","Data":"3592b4d3ad679a38808b14653e8190ac33599bbf6a05ee295ea0966a7fd4c55c"}
Jul 28 19:35:53.104232 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.104201    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:35:53.119486 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.119445    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-rh26c" podStartSLOduration=66.119431179 podStartE2EDuration="1m6.119431179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:53.119374829 +0000 UTC m=+65.930936711" watchObservedRunningTime="2026-07-28 19:35:53.119431179 +0000 UTC m=+65.930993041"
Jul 28 19:35:53.260745 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.260649    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:53.652957 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.652922    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:53.670983 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.670856    2568 scope.go:117] "RemoveContainer" containerID="ba4009f402cf6a7da7230e201de1b0baee72870d569c719b2e189432c3bbbdbb"
Jul 28 19:35:53.855469 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.855301    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:53.913811 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:53.913730    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:54.058699 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.058667    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:54.110729 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.110693    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hs6df" event={"ID":"967eb3cb-a574-4af5-bba1-78655a105b62","Type":"ContainerStarted","Data":"92d4c19b7c82cd9cd670dc0dbb850d4ca648366af9db14907b399ff5867e57ec"}
Jul 28 19:35:54.111165 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.110794    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-hs6df" event={"ID":"967eb3cb-a574-4af5-bba1-78655a105b62","Type":"ContainerStarted","Data":"fec4420c0dec9708bf6d77e811b49179f0a8ddb479ca0bbf909f8543145c88b3"}
Jul 28 19:35:54.113145 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.113124    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:35:54.113287 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.113237    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns" event={"ID":"52f051cf-a638-429d-b7e1-3091504ad33e","Type":"ContainerStarted","Data":"5c7ec7136f263b42bfc629edf168338a6c299ea194445dbb69d501c2d9170600"}
Jul 28 19:35:54.113947 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.113818    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:54.148456 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.148410    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns" podStartSLOduration=17.740285489 podStartE2EDuration="24.148390439s" totalImagesPullingTime="6.40810495s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:54.148221173 +0000 UTC m=+66.959783064" watchObservedRunningTime="2026-07-28 19:35:54.148390439 +0000 UTC m=+66.959952305"
Jul 28 19:35:54.149050 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.148766    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-hs6df" podStartSLOduration=66.043470359 podStartE2EDuration="1m7.148752735s" totalImagesPullingTime="1.105282376s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:54.129752806 +0000 UTC m=+66.941314670" watchObservedRunningTime="2026-07-28 19:35:54.148752735 +0000 UTC m=+66.960314599"
Jul 28 19:35:54.462896 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.462848    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:54.587768 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.587733    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:54.587947 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.587878    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:54.590186 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.590158    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/9ce1caf2-c10b-4415-9058-7688ea24cafd-metrics-tls\") pod \"dns-default-gzdh7\" (UID: \"9ce1caf2-c10b-4415-9058-7688ea24cafd\") " pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:54.590341 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.590304    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"image-registry-6db6c45f8b-r74tv\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") " pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:54.743738 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.743658    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-9p5kr\""
Jul 28 19:35:54.751679 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.751648    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:54.756098 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.756074    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-lt6xm\""
Jul 28 19:35:54.764457 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.764096    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:35:54.790695 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.790646    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:54.790845 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.790738    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:54.791606 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.791584    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a5224743-992f-4d1c-86fe-750213fb90f4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:54.794137 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.794023    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a5224743-992f-4d1c-86fe-750213fb90f4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mqc6s\" (UID: \"a5224743-992f-4d1c-86fe-750213fb90f4\") " pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:54.805264 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.805133    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-584dcb6b54-9zrns"
Jul 28 19:35:54.841499 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.841223    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-rtz66\""
Jul 28 19:35:54.850350 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.848787    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mqc6s"
Jul 28 19:35:54.939558 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.939483    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6db6c45f8b-r74tv"]
Jul 28 19:35:54.942280 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:54.942246    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c14f47a_1bfb_4c6c_94a2_46f6acfd364c.slice/crio-650ca0b8c365f80d9e92de5c57066a46f430e94aa8651f3f31bec6b39782e165 WatchSource:0}: Error finding container 650ca0b8c365f80d9e92de5c57066a46f430e94aa8651f3f31bec6b39782e165: Status 404 returned error can't find the container with id 650ca0b8c365f80d9e92de5c57066a46f430e94aa8651f3f31bec6b39782e165
Jul 28 19:35:54.952839 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.952816    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gzdh7"]
Jul 28 19:35:54.993391 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:54.993355    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:55.032210 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.032176    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mqc6s"]
Jul 28 19:35:55.036046 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:35:55.036018    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5224743_992f_4d1c_86fe_750213fb90f4.slice/crio-82483b5d224a8bbbf24ecdf4ef8a804a9c80aec38e3ddf26f272d260bd292ce2 WatchSource:0}: Error finding container 82483b5d224a8bbbf24ecdf4ef8a804a9c80aec38e3ddf26f272d260bd292ce2: Status 404 returned error can't find the container with id 82483b5d224a8bbbf24ecdf4ef8a804a9c80aec38e3ddf26f272d260bd292ce2
Jul 28 19:35:55.093216 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.093180    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:55.118956 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.118912    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv" event={"ID":"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c","Type":"ContainerStarted","Data":"f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971"}
Jul 28 19:35:55.119478 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.118974    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv" event={"ID":"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c","Type":"ContainerStarted","Data":"650ca0b8c365f80d9e92de5c57066a46f430e94aa8651f3f31bec6b39782e165"}
Jul 28 19:35:55.119478 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.119059    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:35:55.120523 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.120491    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mqc6s" event={"ID":"a5224743-992f-4d1c-86fe-750213fb90f4","Type":"ContainerStarted","Data":"82483b5d224a8bbbf24ecdf4ef8a804a9c80aec38e3ddf26f272d260bd292ce2"}
Jul 28 19:35:55.121681 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.121657    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gzdh7" event={"ID":"9ce1caf2-c10b-4415-9058-7688ea24cafd","Type":"ContainerStarted","Data":"91acc6e41e90f044b142e4b587788e2072d37d377fd8b071897bf8ae9108b388"}
Jul 28 19:35:55.140096 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.139921    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv" podStartSLOduration=67.139904021 podStartE2EDuration="1m7.139904021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:34:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:35:55.138524526 +0000 UTC m=+67.950086414" watchObservedRunningTime="2026-07-28 19:35:55.139904021 +0000 UTC m=+67.951465885"
Jul 28 19:35:55.282366 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.282263    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:55.664265 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.664181    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:55.724527 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.724473    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:55.904766 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:55.904571    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:56.709477 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:56.709436    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:56.768554 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:56.768464    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:56.879169 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:56.878906    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:57.079190 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:57.079025    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:57.696822 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:57.696786    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:35:57.797975 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:35:57.797924    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:01.150429 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:01.150385    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6w927" event={"ID":"66ebf5f5-ab7c-4cb7-9857-b376b202edae","Type":"ContainerStarted","Data":"0aa19ed46633d0f59c8626b6a7219b2b3fd908fe91b3f160ab8eeaba4e34fe7a"}
Jul 28 19:36:01.154073 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:01.152561    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mqc6s" event={"ID":"a5224743-992f-4d1c-86fe-750213fb90f4","Type":"ContainerStarted","Data":"531492a9815d25e649d33729eadfc4ee79b2dec5ee263a7c8abe394df88a6843"}
Jul 28 19:36:01.155397 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:01.155256    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gzdh7" event={"ID":"9ce1caf2-c10b-4415-9058-7688ea24cafd","Type":"ContainerStarted","Data":"16784a49716827beb261544c55bdaa3a47ef12665db29ecbf27f5d8e081b5938"}
Jul 28 19:36:01.172088 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:01.172021    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6w927" podStartSLOduration=64.881959834 podStartE2EDuration="1m13.171997608s" totalImagesPullingTime="8.290037774s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:34:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:01.170422489 +0000 UTC m=+73.981984349" watchObservedRunningTime="2026-07-28 19:36:01.171997608 +0000 UTC m=+73.983559471"
Jul 28 19:36:02.161826 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.161783    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gzdh7" event={"ID":"9ce1caf2-c10b-4415-9058-7688ea24cafd","Type":"ContainerStarted","Data":"c87dce747ab2bbdd519e299f21e38ba48e289471b2ccf60edbb712d9b56ecdb0"}
Jul 28 19:36:02.181748 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.181689    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gzdh7" podStartSLOduration=34.184534395 podStartE2EDuration="40.181672644s" totalImagesPullingTime="5.997138249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:02.180184892 +0000 UTC m=+74.991746777" watchObservedRunningTime="2026-07-28 19:36:02.181672644 +0000 UTC m=+74.993234506"
Jul 28 19:36:02.242166 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.242128    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:02.305149 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.305060    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:02.663862 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.663805    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:36:02.669587 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.669537    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-548b489cb8-kgnw9\" (UID: \"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3\") " pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:36:02.670401 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.670372    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"]
Jul 28 19:36:02.678514 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.678494    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"]
Jul 28 19:36:02.678661 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.678639    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.682943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.682923    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"]
Jul 28 19:36:02.683068 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.683014    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"
Jul 28 19:36:02.683992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.683682    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 28 19:36:02.683992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.683724    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 28 19:36:02.683992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.683778    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 28 19:36:02.683992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.683829    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 28 19:36:02.684559 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.684532    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 28 19:36:02.684694 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.684673    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 28 19:36:02.684796 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.684735    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 28 19:36:02.686013 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.685993    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-25ncs\""
Jul 28 19:36:02.687684 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.687666    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.690742 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.690722    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\""
Jul 28 19:36:02.693552 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.693534    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 28 19:36:02.708844 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.708810    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"]
Jul 28 19:36:02.713133 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.713114    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"]
Jul 28 19:36:02.714823 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.714755    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"]
Jul 28 19:36:02.765219 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.765189    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-ca\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.765385 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.765232    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.765385 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.765288    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzltk\" (UniqueName: \"kubernetes.io/projected/900b536e-0572-427c-bbfd-3f6521a9ea8a-kube-api-access-bzltk\") pod \"managed-serviceaccount-addon-agent-f47744cf4-jtwpj\" (UID: \"900b536e-0572-427c-bbfd-3f6521a9ea8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"
Jul 28 19:36:02.765515 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.765428    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/900b536e-0572-427c-bbfd-3f6521a9ea8a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f47744cf4-jtwpj\" (UID: \"900b536e-0572-427c-bbfd-3f6521a9ea8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"
Jul 28 19:36:02.765515 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.765454    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-hub\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.765619 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.765517    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.765619 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.765548    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/126058fe-48fe-4280-98c2-09dc09a3da09-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.765619 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.765611    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7jpt2\" (UniqueName: \"kubernetes.io/projected/126058fe-48fe-4280-98c2-09dc09a3da09-kube-api-access-7jpt2\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.772992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.772965    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-xvlm7"]
Jul 28 19:36:02.780060 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.779870    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-xvlm7"
Jul 28 19:36:02.780291 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.780078    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:02.782507 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.782485    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-8g856\""
Jul 28 19:36:02.782805 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.782779    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Jul 28 19:36:02.782805 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.782798    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Jul 28 19:36:02.794763 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.794736    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-xvlm7"]
Jul 28 19:36:02.856870 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.856839    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-965px\""
Jul 28 19:36:02.863778 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.863755    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"
Jul 28 19:36:02.866398 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866289    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/900b536e-0572-427c-bbfd-3f6521a9ea8a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f47744cf4-jtwpj\" (UID: \"900b536e-0572-427c-bbfd-3f6521a9ea8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"
Jul 28 19:36:02.866398 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866348    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-hub\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866630    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjwsx\" (UniqueName: \"kubernetes.io/projected/399fc0cd-afef-48e4-9468-05e6f5037042-kube-api-access-rjwsx\") pod \"downloads-7c97dc4555-xvlm7\" (UID: \"399fc0cd-afef-48e4-9468-05e6f5037042\") " pod="openshift-console/downloads-7c97dc4555-xvlm7"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866676    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866759    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866807    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/126058fe-48fe-4280-98c2-09dc09a3da09-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866884    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7jpt2\" (UniqueName: \"kubernetes.io/projected/126058fe-48fe-4280-98c2-09dc09a3da09-kube-api-access-7jpt2\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866948    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba03067f-e48d-43ad-b777-e0c16fa85bc6-tmp\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.866986    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.867011    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-ca\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.867037    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ba03067f-e48d-43ad-b777-e0c16fa85bc6-klusterlet-config\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.867071    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.867098    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzltk\" (UniqueName: \"kubernetes.io/projected/900b536e-0572-427c-bbfd-3f6521a9ea8a-kube-api-access-bzltk\") pod \"managed-serviceaccount-addon-agent-f47744cf4-jtwpj\" (UID: \"900b536e-0572-427c-bbfd-3f6521a9ea8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"
Jul 28 19:36:02.867208 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.867127    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cqx87\" (UniqueName: \"kubernetes.io/projected/ba03067f-e48d-43ad-b777-e0c16fa85bc6-kube-api-access-cqx87\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.867917 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.867671    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/126058fe-48fe-4280-98c2-09dc09a3da09-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.869864 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.869276    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-service-ca-bundle\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:02.869864 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.869821    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-hub\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.870446 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.870053    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/e9190e9f-9c61-4c82-aa8a-b14cc3e1c348-metrics-certs\") pod \"router-default-5d9bfc7fdf-nmlwq\" (UID: \"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348\") " pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:02.870586 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.870563    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/900b536e-0572-427c-bbfd-3f6521a9ea8a-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-f47744cf4-jtwpj\" (UID: \"900b536e-0572-427c-bbfd-3f6521a9ea8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"
Jul 28 19:36:02.871258 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.871236    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-ca\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.871486 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.871427    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.871672 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.871543    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:02.872745 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.872722    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/126058fe-48fe-4280-98c2-09dc09a3da09-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.881767 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.881740    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzltk\" (UniqueName: \"kubernetes.io/projected/900b536e-0572-427c-bbfd-3f6521a9ea8a-kube-api-access-bzltk\") pod \"managed-serviceaccount-addon-agent-f47744cf4-jtwpj\" (UID: \"900b536e-0572-427c-bbfd-3f6521a9ea8a\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"
Jul 28 19:36:02.881871 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.881806    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7jpt2\" (UniqueName: \"kubernetes.io/projected/126058fe-48fe-4280-98c2-09dc09a3da09-kube-api-access-7jpt2\") pod \"cluster-proxy-proxy-agent-59f459f786-dv2j2\" (UID: \"126058fe-48fe-4280-98c2-09dc09a3da09\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:02.968017 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.967929    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:02.968401 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.968375    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjwsx\" (UniqueName: \"kubernetes.io/projected/399fc0cd-afef-48e4-9468-05e6f5037042-kube-api-access-rjwsx\") pod \"downloads-7c97dc4555-xvlm7\" (UID: \"399fc0cd-afef-48e4-9468-05e6f5037042\") " pod="openshift-console/downloads-7c97dc4555-xvlm7"
Jul 28 19:36:02.968554 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.968471    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba03067f-e48d-43ad-b777-e0c16fa85bc6-tmp\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.968554 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.968510    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ba03067f-e48d-43ad-b777-e0c16fa85bc6-klusterlet-config\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.969012 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.968963    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cqx87\" (UniqueName: \"kubernetes.io/projected/ba03067f-e48d-43ad-b777-e0c16fa85bc6-kube-api-access-cqx87\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.969384 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.969365    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ba03067f-e48d-43ad-b777-e0c16fa85bc6-tmp\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.974728 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.971901    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ba03067f-e48d-43ad-b777-e0c16fa85bc6-klusterlet-config\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:02.978855 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.978832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjwsx\" (UniqueName: \"kubernetes.io/projected/399fc0cd-afef-48e4-9468-05e6f5037042-kube-api-access-rjwsx\") pod \"downloads-7c97dc4555-xvlm7\" (UID: \"399fc0cd-afef-48e4-9468-05e6f5037042\") " pod="openshift-console/downloads-7c97dc4555-xvlm7"
Jul 28 19:36:02.979187 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:02.979165    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cqx87\" (UniqueName: \"kubernetes.io/projected/ba03067f-e48d-43ad-b777-e0c16fa85bc6-kube-api-access-cqx87\") pod \"klusterlet-addon-workmgr-54c67b7666-wllkm\" (UID: \"ba03067f-e48d-43ad-b777-e0c16fa85bc6\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:03.005808 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.005781    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"
Jul 28 19:36:03.014492 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.014472    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"
Jul 28 19:36:03.023345 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.023311    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-z7j6f\""
Jul 28 19:36:03.031148 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.031113    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:03.031241 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.031161    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:03.069069 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.069036    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:03.097509 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.097482    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-xvlm7"
Jul 28 19:36:03.166669 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.166637    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:36:03.169039 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:03.168953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:04.311977 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:04.311928    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:04.916801 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:04.916727    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-5d9bfc7fdf-nmlwq"]
Jul 28 19:36:05.075920 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:05.075851    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9190e9f_9c61_4c82_aa8a_b14cc3e1c348.slice/crio-5726e491a842b7799ebb778f2f3e7c71ee686cdd54b3b2e3fb6afe9ee09b3943 WatchSource:0}: Error finding container 5726e491a842b7799ebb778f2f3e7c71ee686cdd54b3b2e3fb6afe9ee09b3943: Status 404 returned error can't find the container with id 5726e491a842b7799ebb778f2f3e7c71ee686cdd54b3b2e3fb6afe9ee09b3943
Jul 28 19:36:05.139867 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.139840    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-xvlm7"]
Jul 28 19:36:05.143145 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.143045    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2"]
Jul 28 19:36:05.143961 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:05.143934    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod399fc0cd_afef_48e4_9468_05e6f5037042.slice/crio-8db95c8eec053b5d92159364f58e286ac3a6559fd340ea7e9788d88386d28d26 WatchSource:0}: Error finding container 8db95c8eec053b5d92159364f58e286ac3a6559fd340ea7e9788d88386d28d26: Status 404 returned error can't find the container with id 8db95c8eec053b5d92159364f58e286ac3a6559fd340ea7e9788d88386d28d26
Jul 28 19:36:05.146115 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:05.146089    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod126058fe_48fe_4280_98c2_09dc09a3da09.slice/crio-347a0e29f9469aca3fab2445cdcf624c253bf31f9e9149c58848786bf7375e01 WatchSource:0}: Error finding container 347a0e29f9469aca3fab2445cdcf624c253bf31f9e9149c58848786bf7375e01: Status 404 returned error can't find the container with id 347a0e29f9469aca3fab2445cdcf624c253bf31f9e9149c58848786bf7375e01
Jul 28 19:36:05.174196 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.174173    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9"]
Jul 28 19:36:05.177056 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.177025    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-xvlm7" event={"ID":"399fc0cd-afef-48e4-9468-05e6f5037042","Type":"ContainerStarted","Data":"8db95c8eec053b5d92159364f58e286ac3a6559fd340ea7e9788d88386d28d26"}
Jul 28 19:36:05.177401 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.177377    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj"]
Jul 28 19:36:05.178476 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.178430    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2" event={"ID":"126058fe-48fe-4280-98c2-09dc09a3da09","Type":"ContainerStarted","Data":"347a0e29f9469aca3fab2445cdcf624c253bf31f9e9149c58848786bf7375e01"}
Jul 28 19:36:05.178739 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.178716    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"]
Jul 28 19:36:05.179549 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:05.179524    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1ace64_f78c_4dc0_ad9e_fdaa8ff64ad3.slice/crio-9343e1b1480e694fa3c31f5d45f4edc3606e83028ceccbd41fd1f105917a5ed9 WatchSource:0}: Error finding container 9343e1b1480e694fa3c31f5d45f4edc3606e83028ceccbd41fd1f105917a5ed9: Status 404 returned error can't find the container with id 9343e1b1480e694fa3c31f5d45f4edc3606e83028ceccbd41fd1f105917a5ed9
Jul 28 19:36:05.179963 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.179932    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq" event={"ID":"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348","Type":"ContainerStarted","Data":"5726e491a842b7799ebb778f2f3e7c71ee686cdd54b3b2e3fb6afe9ee09b3943"}
Jul 28 19:36:05.180665 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:05.180637    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod900b536e_0572_427c_bbfd_3f6521a9ea8a.slice/crio-7865e9cc268ad3b5e4f52f06c86d814b68367e9263b10d74eac3fb415e5c6a60 WatchSource:0}: Error finding container 7865e9cc268ad3b5e4f52f06c86d814b68367e9263b10d74eac3fb415e5c6a60: Status 404 returned error can't find the container with id 7865e9cc268ad3b5e4f52f06c86d814b68367e9263b10d74eac3fb415e5c6a60
Jul 28 19:36:05.181500 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:05.181477    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba03067f_e48d_43ad_b777_e0c16fa85bc6.slice/crio-0f296ace49b67a2dcba4343eaa66fee651f364a2fa02bd2c4f06c29c1bc570f6 WatchSource:0}: Error finding container 0f296ace49b67a2dcba4343eaa66fee651f364a2fa02bd2c4f06c29c1bc570f6: Status 404 returned error can't find the container with id 0f296ace49b67a2dcba4343eaa66fee651f364a2fa02bd2c4f06c29c1bc570f6
Jul 28 19:36:05.921277 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.920549    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5746b8bd47-mnv6q"]
Jul 28 19:36:05.943821 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.943780    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5746b8bd47-mnv6q"]
Jul 28 19:36:05.943989 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.943926    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:05.947181 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.947156    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 19:36:05.947314 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.947228    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 19:36:05.947314 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.947295    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 19:36:05.947314 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.947159    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 19:36:05.947512 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.947164    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 19:36:05.948032 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:05.948013    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dkbk4\""
Jul 28 19:36:06.094374 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.094317    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-serving-cert\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.094552 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.094440    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-config\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.094616 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.094553    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-chx62\" (UniqueName: \"kubernetes.io/projected/c0d96890-64f2-4bd4-b378-a3d79db95fec-kube-api-access-chx62\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.094616 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.094582    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-service-ca\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.094616 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.094609    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-oauth-serving-cert\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.094780 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.094694    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-oauth-config\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.097105 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.097064    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:06.188475 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.187555    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq" event={"ID":"e9190e9f-9c61-4c82-aa8a-b14cc3e1c348","Type":"ContainerStarted","Data":"3333e62c61ca80e7b82c22143e9ce6235aa01a3b4ee03cde94965ba399dd6fd4"}
Jul 28 19:36:06.192596 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.192530    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mqc6s" event={"ID":"a5224743-992f-4d1c-86fe-750213fb90f4","Type":"ContainerStarted","Data":"f446c6f9bb2f4a91e255058fab7cf3d10209d097f699000a2a693d2e5984f563"}
Jul 28 19:36:06.195889 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.195369    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-service-ca\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.195889 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.195411    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-oauth-serving-cert\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.195889 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.195448    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-oauth-config\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.195889 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.195501    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-serving-cert\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.196810 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.196164    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-oauth-serving-cert\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.196810 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.196188    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-service-ca\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.196810 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.196251    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-config\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.196810 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.196336    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-chx62\" (UniqueName: \"kubernetes.io/projected/c0d96890-64f2-4bd4-b378-a3d79db95fec-kube-api-access-chx62\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.197235 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.197169    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-config\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.198751 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.198373    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj" event={"ID":"900b536e-0572-427c-bbfd-3f6521a9ea8a","Type":"ContainerStarted","Data":"7865e9cc268ad3b5e4f52f06c86d814b68367e9263b10d74eac3fb415e5c6a60"}
Jul 28 19:36:06.200179 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.199990    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:06.200740 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.200586    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm" event={"ID":"ba03067f-e48d-43ad-b777-e0c16fa85bc6","Type":"ContainerStarted","Data":"0f296ace49b67a2dcba4343eaa66fee651f364a2fa02bd2c4f06c29c1bc570f6"}
Jul 28 19:36:06.202920 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.202874    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-serving-cert\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.203095 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.203073    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9" event={"ID":"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3","Type":"ContainerStarted","Data":"9343e1b1480e694fa3c31f5d45f4edc3606e83028ceccbd41fd1f105917a5ed9"}
Jul 28 19:36:06.210356 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.209986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-chx62\" (UniqueName: \"kubernetes.io/projected/c0d96890-64f2-4bd4-b378-a3d79db95fec-kube-api-access-chx62\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.210356 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.210282    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-oauth-config\") pod \"console-5746b8bd47-mnv6q\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") " pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.210849 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.210802    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq" podStartSLOduration=36.210784243 podStartE2EDuration="36.210784243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:06.210233416 +0000 UTC m=+79.021795307" watchObservedRunningTime="2026-07-28 19:36:06.210784243 +0000 UTC m=+79.022346110"
Jul 28 19:36:06.257717 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.257690    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:06.462867 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:06.462784    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5746b8bd47-mnv6q"]
Jul 28 19:36:06.468000 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:06.467968    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0d96890_64f2_4bd4_b378_a3d79db95fec.slice/crio-3d06194f34c157c57e5dc1ab1938fbdf3ed6725e9f78c5ca5c35aad31dea07f6 WatchSource:0}: Error finding container 3d06194f34c157c57e5dc1ab1938fbdf3ed6725e9f78c5ca5c35aad31dea07f6: Status 404 returned error can't find the container with id 3d06194f34c157c57e5dc1ab1938fbdf3ed6725e9f78c5ca5c35aad31dea07f6
Jul 28 19:36:07.005837 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.005789    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:36:07.024493 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.024436    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e5087f7a-f48f-4dc9-a7ac-d1a18643367b-networking-console-plugin-cert\") pod \"networking-console-plugin-74c9f884b7-lw2hv\" (UID: \"e5087f7a-f48f-4dc9-a7ac-d1a18643367b\") " pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:36:07.032529 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.032454    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:07.035933 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.035605    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:07.245431 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.245393    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5746b8bd47-mnv6q" event={"ID":"c0d96890-64f2-4bd4-b378-a3d79db95fec","Type":"ContainerStarted","Data":"3d06194f34c157c57e5dc1ab1938fbdf3ed6725e9f78c5ca5c35aad31dea07f6"}
Jul 28 19:36:07.246463 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.246367    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:07.247884 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.247863    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-5d9bfc7fdf-nmlwq"
Jul 28 19:36:07.273290 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.273057    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-55bmj\""
Jul 28 19:36:07.283377 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.281703    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"
Jul 28 19:36:07.537959 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:07.537910    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv"]
Jul 28 19:36:07.542110 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:07.542078    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5087f7a_f48f_4dc9_a7ac_d1a18643367b.slice/crio-f7593d5d7c1348d38cd20530c15a0ca27d7f9d940e0def218137906ed6305820 WatchSource:0}: Error finding container f7593d5d7c1348d38cd20530c15a0ca27d7f9d940e0def218137906ed6305820: Status 404 returned error can't find the container with id f7593d5d7c1348d38cd20530c15a0ca27d7f9d940e0def218137906ed6305820
Jul 28 19:36:08.115344 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:08.115160    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:08.177344 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:08.177272    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:08.278379 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:08.278260    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv" event={"ID":"e5087f7a-f48f-4dc9-a7ac-d1a18643367b","Type":"ContainerStarted","Data":"f7593d5d7c1348d38cd20530c15a0ca27d7f9d940e0def218137906ed6305820"}
Jul 28 19:36:08.455271 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:08.455128    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:10.537553 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:10.537127    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:13.173721 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:13.173687    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gzdh7"
Jul 28 19:36:14.120885 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:14.120842    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:14.221201 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:14.221157    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:14.294971 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:14.294933    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:14.701423 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:14.701184    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:14.758858 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:14.758814    2568 patch_prober.go:28] interesting pod/image-registry-6db6c45f8b-r74tv container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Jul 28 19:36:14.759037 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:14.758888    2568 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv" podUID="9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 28 19:36:14.801438 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:14.801256    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:15.043876 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:15.043780    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:15.103837 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:15.103794    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:16.135497 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:16.135465    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:36:16.397255 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:16.397221    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:16.453864 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:16.453828    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:17.351339 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:17.351298    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:17.416102 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:17.415999    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:17.705814 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:17.705781    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:17.807401 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:17.807229    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:17.972371 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:17.972254    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:18.025562 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:18.025528    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:18.079639 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:18.079567    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:20.503316 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:20.503272    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:20.604379 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:20.604347    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:21.697274 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:21.697238    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:21.799576 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:21.799486    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:22.362266 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:22.362232    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:22.513891 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:22.513846    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:24.116178 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:24.116144    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-rh26c"
Jul 28 19:36:24.169430 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:24.169381    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:24.243252 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:24.243219    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:25.069900 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:25.069861    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5746b8bd47-mnv6q"]
Jul 28 19:36:25.145136 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:25.145105    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:25.235978 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:25.235943    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:25.335231 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:25.335142    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:25.436404 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:25.436369    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:25.578660 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:25.578622    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:25.667347 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:25.667306    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:26.047868 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.047819    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:26.148806 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.148761    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:26.240002 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.239924    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:26.308235 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.308192    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:26.388271 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.388151    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv" event={"ID":"e5087f7a-f48f-4dc9-a7ac-d1a18643367b","Type":"ContainerStarted","Data":"2e35a396de4fe4cd7410bc4ffe5da5c654c158372816e7fe3ffa751a4b626cca"}
Jul 28 19:36:26.391225 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.391173    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mqc6s" event={"ID":"a5224743-992f-4d1c-86fe-750213fb90f4","Type":"ContainerStarted","Data":"9daa243daf59364c6fd9b7a75fe5664d0fa76dd0ba0a8585342fbf84f3e2da35"}
Jul 28 19:36:26.393094 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.393008    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj" event={"ID":"900b536e-0572-427c-bbfd-3f6521a9ea8a","Type":"ContainerStarted","Data":"311a4071c29f823e135445c6eb663d04ea0b446873e1777a6a38506dfc623b50"}
Jul 28 19:36:26.394891 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.394864    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm" event={"ID":"ba03067f-e48d-43ad-b777-e0c16fa85bc6","Type":"ContainerStarted","Data":"6c23e10c0c49240b98516563d92cdc63c8fe1a638b85d704f239c8c20c18b2b6"}
Jul 28 19:36:26.395081 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.395037    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:26.397078 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.396844    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9" event={"ID":"0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3","Type":"ContainerStarted","Data":"864a12a8a2e103adcfdddbb103bfda7757d5d4ed78696a5ce38a26daaf1a2ef5"}
Jul 28 19:36:26.397180 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.397093    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm"
Jul 28 19:36:26.399309 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.399026    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-xvlm7" event={"ID":"399fc0cd-afef-48e4-9468-05e6f5037042","Type":"ContainerStarted","Data":"9f01bf816eeb1a0bcebb650eaa782b5791596bb98e6d974ccf0a7f59a97b9ff8"}
Jul 28 19:36:26.399309 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.399273    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-xvlm7"
Jul 28 19:36:26.401470 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.401347    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5746b8bd47-mnv6q" event={"ID":"c0d96890-64f2-4bd4-b378-a3d79db95fec","Type":"ContainerStarted","Data":"affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9"}
Jul 28 19:36:26.404770 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.404745    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2" event={"ID":"126058fe-48fe-4280-98c2-09dc09a3da09","Type":"ContainerStarted","Data":"a681858b305c9d9acc710be16ac385412ae0d5035cbe851095ccfbd9b6ab27a6"}
Jul 28 19:36:26.404873 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.404786    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2" event={"ID":"126058fe-48fe-4280-98c2-09dc09a3da09","Type":"ContainerStarted","Data":"880c753b59189c9501f4f7acbf69ef87c4f4a69b64b43a3e6770c7f12412a3d0"}
Jul 28 19:36:26.404873 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.404804    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2" event={"ID":"126058fe-48fe-4280-98c2-09dc09a3da09","Type":"ContainerStarted","Data":"a3e143daa2c92b17fb4d27927cebb51ebf13de13bb115443c31ee535921a93ad"}
Jul 28 19:36:26.405154 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.405112    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-74c9f884b7-lw2hv" podStartSLOduration=40.576006877 podStartE2EDuration="51.405097102s" totalImagesPullingTime="10.829090225s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:26.404437604 +0000 UTC m=+99.215999470" watchObservedRunningTime="2026-07-28 19:36:26.405097102 +0000 UTC m=+99.216658966"
Jul 28 19:36:26.411444 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.411423    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-xvlm7"
Jul 28 19:36:26.441663 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.441601    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-f47744cf4-jtwpj" podStartSLOduration=11.250456123 podStartE2EDuration="24.441579701s" totalImagesPullingTime="13.191123578s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:26.440946696 +0000 UTC m=+99.252508560" watchObservedRunningTime="2026-07-28 19:36:26.441579701 +0000 UTC m=+99.253141565"
Jul 28 19:36:26.442586 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.442543    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-xvlm7" podStartSLOduration=3.632249002 podStartE2EDuration="24.442529407s" totalImagesPullingTime="20.810280405s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:26.425498138 +0000 UTC m=+99.237060004" watchObservedRunningTime="2026-07-28 19:36:26.442529407 +0000 UTC m=+99.254091270"
Jul 28 19:36:26.478556 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.478500    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-548b489cb8-kgnw9" podStartSLOduration=43.336044135 podStartE2EDuration="56.478481385s" totalImagesPullingTime="13.14243725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:35:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:26.478392194 +0000 UTC m=+99.289954058" watchObservedRunningTime="2026-07-28 19:36:26.478481385 +0000 UTC m=+99.290043250"
Jul 28 19:36:26.479781 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.479739    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-54c67b7666-wllkm" podStartSLOduration=11.289703516 podStartE2EDuration="24.47972589s" totalImagesPullingTime="13.190022374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:26.460245403 +0000 UTC m=+99.271807266" watchObservedRunningTime="2026-07-28 19:36:26.47972589 +0000 UTC m=+99.291287779"
Jul 28 19:36:26.497459 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.497377    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5746b8bd47-mnv6q" podStartSLOduration=2.310420295 podStartE2EDuration="21.497363098s" totalImagesPullingTime="19.186942803s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:26.496185813 +0000 UTC m=+99.307747702" watchObservedRunningTime="2026-07-28 19:36:26.497363098 +0000 UTC m=+99.308924965"
Jul 28 19:36:26.509948 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.509923    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:26.518711 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.518665    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mqc6s" podStartSLOduration=25.381885757 podStartE2EDuration="48.518644582s" totalImagesPullingTime="23.136758825s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 19:35:38 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:26.518544038 +0000 UTC m=+99.330105929" watchObservedRunningTime="2026-07-28 19:36:26.518644582 +0000 UTC m=+99.330206449"
Jul 28 19:36:26.542884 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.542830    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59f459f786-dv2j2" podStartSLOduration=11.296882491 podStartE2EDuration="24.542815973s" totalImagesPullingTime="13.245933482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:36:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:26.540955743 +0000 UTC m=+99.352517597" watchObservedRunningTime="2026-07-28 19:36:26.542815973 +0000 UTC m=+99.354377837"
Jul 28 19:36:26.609159 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:26.609123    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:29.240762 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:29.240719    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:29.311050 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:29.311005    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:29.979996 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:29.979954    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:30.041939 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:30.041900    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6db6c45f8b-r74tv"]
Jul 28 19:36:30.130806 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:30.130765    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:30.231148 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:30.231061    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:31.860448 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:31.860413    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:32.247749 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:32.247713    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:32.327333 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:32.327286    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:32.606082 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:32.605991    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:32.665512 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:32.665476    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:33.393705 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.393663    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:33.596449 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.596410    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:33.652756 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.652674    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-8hccr"]
Jul 28 19:36:33.690725 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.690689    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:33.719848 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.719814    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.723581 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.723536    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 28 19:36:33.724376 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.724356    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 28 19:36:33.724468 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.724411    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 28 19:36:33.724468 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.724443    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-8dtwl\""
Jul 28 19:36:33.724577 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.724564    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 28 19:36:33.782283 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.782250    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:33.856024 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.855988    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-root\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.856195 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.856052    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-textfile\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.856195 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.856084    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nsgk\" (UniqueName: \"kubernetes.io/projected/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-kube-api-access-6nsgk\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.856195 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.856107    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-accelerators-collector-config\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.856371 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.856217    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.856371 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.856271    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-wtmp\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.856371 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.856310    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-tls\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.856515 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.856380    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-sys\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.856515 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.856403    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-metrics-client-ca\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.884968 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.884935    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:33.957528 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957445    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-tls\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.957697 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957603    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-sys\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.957697 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957636    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-metrics-client-ca\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.957697 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957668    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-root\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.957861 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957715    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-textfile\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.957861 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957745    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nsgk\" (UniqueName: \"kubernetes.io/projected/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-kube-api-access-6nsgk\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.957861 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957774    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-accelerators-collector-config\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.957861 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957841    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.958065 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.957887    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-wtmp\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.958065 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.958046    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-wtmp\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.958216 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.958192    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-textfile\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.958435 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.958400    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-root\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.958532 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.958513    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-sys\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.958743 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.958721    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-metrics-client-ca\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.958883 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.958774    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-accelerators-collector-config\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.961140 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.961023    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.961238 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.961186    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-node-exporter-tls\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.966480 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.966453    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nsgk\" (UniqueName: \"kubernetes.io/projected/3dbc7762-8ad9-4a4a-b513-1a3aba7f845c-kube-api-access-6nsgk\") pod \"node-exporter-8hccr\" (UID: \"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c\") " pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:33.983135 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:33.983108    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:34.033566 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.033534    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-8hccr"
Jul 28 19:36:34.060224 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.060192    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:34.083785 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:36:34.083749    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3dbc7762_8ad9_4a4a_b513_1a3aba7f845c.slice/crio-9415e9d46e32e8b9dbd7907d3185e7e7fcb97dfc140e9a473882cce670eb4e81 WatchSource:0}: Error finding container 9415e9d46e32e8b9dbd7907d3185e7e7fcb97dfc140e9a473882cce670eb4e81: Status 404 returned error can't find the container with id 9415e9d46e32e8b9dbd7907d3185e7e7fcb97dfc140e9a473882cce670eb4e81
Jul 28 19:36:34.127048 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.127007    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:34.223942 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.223867    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:34.323802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.323769    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:34.423302 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.423265    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:34.445135 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.445099    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8hccr" event={"ID":"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c","Type":"ContainerStarted","Data":"9415e9d46e32e8b9dbd7907d3185e7e7fcb97dfc140e9a473882cce670eb4e81"}
Jul 28 19:36:34.650644 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.650260    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:34.733105 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:34.732950    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:35.977898 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:35.977870    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:36.258722 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:36.258650    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:36.457290 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:36.457255    2568 generic.go:356] "Generic (PLEG): container finished" podID="3dbc7762-8ad9-4a4a-b513-1a3aba7f845c" containerID="e0dcaefb6a3df41ef33bf67c68d2d334d20682de9ebc9d662e0da7f304bc8da4" exitCode=0
Jul 28 19:36:36.457463 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:36.457311    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8hccr" event={"ID":"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c","Type":"ContainerDied","Data":"e0dcaefb6a3df41ef33bf67c68d2d334d20682de9ebc9d662e0da7f304bc8da4"}
Jul 28 19:36:36.624015 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:36.623988    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:36.696799 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:36.696769    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:36.829954 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:36.829876    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:37.057214 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:37.057175    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:37.166772 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:37.166733    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:37.237226 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:37.237193    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:37.464169 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:37.464092    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8hccr" event={"ID":"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c","Type":"ContainerStarted","Data":"e483249a22fb0d517060d09d98b020a9885a94ce97f6a8f691e5f10eaa4dba8d"}
Jul 28 19:36:37.464169 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:37.464153    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-8hccr" event={"ID":"3dbc7762-8ad9-4a4a-b513-1a3aba7f845c","Type":"ContainerStarted","Data":"538b968162b96e418ac5315933ce39e01cd7397ac844241a1c20eec00d2453ee"}
Jul 28 19:36:37.486129 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:37.486073    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-8hccr" podStartSLOduration=2.848093873 podStartE2EDuration="4.486056399s" totalImagesPullingTime="1.535673573s" totalInitContainerRuntime="102.288953ms" isStatefulPod=true podCreationTimestamp="2026-07-28 19:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:36:37.48461503 +0000 UTC m=+110.296176893" watchObservedRunningTime="2026-07-28 19:36:37.486056399 +0000 UTC m=+110.297618262"
Jul 28 19:36:38.496032 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:38.495997    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:38.569803 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:38.569764    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:38.671410 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:38.671372    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:39.125050 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:39.125015    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:39.903108 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:39.903073    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:41.683495 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:41.683460    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:41.786502 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:41.786469    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:43.064736 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:43.064702    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:44.494283 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:44.494246    2568 generic.go:356] "Generic (PLEG): container finished" podID="192fad1c-0712-4330-b39f-a094910a4770" containerID="cc8cb489242a0120ace99a8870dc9547744bd0be9ec4cc38030ab21153bc71dd" exitCode=0
Jul 28 19:36:44.494665 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:44.494335    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79" event={"ID":"192fad1c-0712-4330-b39f-a094910a4770","Type":"ContainerDied","Data":"cc8cb489242a0120ace99a8870dc9547744bd0be9ec4cc38030ab21153bc71dd"}
Jul 28 19:36:44.494665 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:44.494653    2568 scope.go:117] "RemoveContainer" containerID="cc8cb489242a0120ace99a8870dc9547744bd0be9ec4cc38030ab21153bc71dd"
Jul 28 19:36:45.500119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:45.500085    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-6f4d6fb6bc-9vl79" event={"ID":"192fad1c-0712-4330-b39f-a094910a4770","Type":"ContainerStarted","Data":"ce9d4167ebe91f27cb1b3a35f4e07af0fed07fcaf00504a7398a70388507c510"}
Jul 28 19:36:47.264239 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:47.264199    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:47.367719 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:47.367685    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:51.425950 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.425886    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-5746b8bd47-mnv6q" podUID="c0d96890-64f2-4bd4-b378-a3d79db95fec" containerName="console" containerID="cri-o://affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9" gracePeriod=15
Jul 28 19:36:51.717119 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.717095    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:51.818924 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.818889    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-oauth-config\") pod \"c0d96890-64f2-4bd4-b378-a3d79db95fec\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") "
Jul 28 19:36:51.819093 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.818932    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-chx62\" (UniqueName: \"kubernetes.io/projected/c0d96890-64f2-4bd4-b378-a3d79db95fec-kube-api-access-chx62\") pod \"c0d96890-64f2-4bd4-b378-a3d79db95fec\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") "
Jul 28 19:36:51.819093 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.818967    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-oauth-serving-cert\") pod \"c0d96890-64f2-4bd4-b378-a3d79db95fec\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") "
Jul 28 19:36:51.819093 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.818990    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-service-ca\") pod \"c0d96890-64f2-4bd4-b378-a3d79db95fec\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") "
Jul 28 19:36:51.819093 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.819021    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-config\") pod \"c0d96890-64f2-4bd4-b378-a3d79db95fec\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") "
Jul 28 19:36:51.819093 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.819070    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-serving-cert\") pod \"c0d96890-64f2-4bd4-b378-a3d79db95fec\" (UID: \"c0d96890-64f2-4bd4-b378-a3d79db95fec\") "
Jul 28 19:36:51.819496 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.819461    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-service-ca" (OuterVolumeSpecName: "service-ca") pod "c0d96890-64f2-4bd4-b378-a3d79db95fec" (UID: "c0d96890-64f2-4bd4-b378-a3d79db95fec"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:51.819608 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.819467    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-config" (OuterVolumeSpecName: "console-config") pod "c0d96890-64f2-4bd4-b378-a3d79db95fec" (UID: "c0d96890-64f2-4bd4-b378-a3d79db95fec"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:51.819608 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.819473    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c0d96890-64f2-4bd4-b378-a3d79db95fec" (UID: "c0d96890-64f2-4bd4-b378-a3d79db95fec"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:51.821277 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.821248    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c0d96890-64f2-4bd4-b378-a3d79db95fec" (UID: "c0d96890-64f2-4bd4-b378-a3d79db95fec"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:36:51.821375 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.821285    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c0d96890-64f2-4bd4-b378-a3d79db95fec" (UID: "c0d96890-64f2-4bd4-b378-a3d79db95fec"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:36:51.821375 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.821316    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0d96890-64f2-4bd4-b378-a3d79db95fec-kube-api-access-chx62" (OuterVolumeSpecName: "kube-api-access-chx62") pod "c0d96890-64f2-4bd4-b378-a3d79db95fec" (UID: "c0d96890-64f2-4bd4-b378-a3d79db95fec"). InnerVolumeSpecName "kube-api-access-chx62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:36:51.919697 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.919665    2568 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-serving-cert\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:51.919697 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.919693    2568 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-oauth-config\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:51.919697 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.919703    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-chx62\" (UniqueName: \"kubernetes.io/projected/c0d96890-64f2-4bd4-b378-a3d79db95fec-kube-api-access-chx62\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:51.919907 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.919712    2568 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-oauth-serving-cert\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:51.919907 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.919722    2568 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:51.919907 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:51.919730    2568 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c0d96890-64f2-4bd4-b378-a3d79db95fec-console-config\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:52.533718 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.533681    2568 generic.go:356] "Generic (PLEG): container finished" podID="c0d96890-64f2-4bd4-b378-a3d79db95fec" containerID="affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9" exitCode=0
Jul 28 19:36:52.534142 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.533722    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5746b8bd47-mnv6q" event={"ID":"c0d96890-64f2-4bd4-b378-a3d79db95fec","Type":"ContainerDied","Data":"affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9"}
Jul 28 19:36:52.534142 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.533739    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5746b8bd47-mnv6q"
Jul 28 19:36:52.534142 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.533784    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5746b8bd47-mnv6q" event={"ID":"c0d96890-64f2-4bd4-b378-a3d79db95fec","Type":"ContainerDied","Data":"3d06194f34c157c57e5dc1ab1938fbdf3ed6725e9f78c5ca5c35aad31dea07f6"}
Jul 28 19:36:52.534142 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.533803    2568 scope.go:117] "RemoveContainer" containerID="affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9"
Jul 28 19:36:52.542115 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.542093    2568 scope.go:117] "RemoveContainer" containerID="affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9"
Jul 28 19:36:52.542371 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:36:52.542350    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9\": container with ID starting with affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9 not found: ID does not exist" containerID="affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9"
Jul 28 19:36:52.542440 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.542379    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9"} err="failed to get container status \"affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9\": rpc error: code = NotFound desc = could not find container \"affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9\": container with ID starting with affbc67579efb2edac0bc8b39cd78ff039b2c485b4198302ee3963adb94ba0e9 not found: ID does not exist"
Jul 28 19:36:52.555188 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.555160    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5746b8bd47-mnv6q"]
Jul 28 19:36:52.558387 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:52.558368    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5746b8bd47-mnv6q"]
Jul 28 19:36:53.675618 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:53.675582    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0d96890-64f2-4bd4-b378-a3d79db95fec" path="/var/lib/kubelet/pods/c0d96890-64f2-4bd4-b378-a3d79db95fec/volumes"
Jul 28 19:36:55.066080 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.066034    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv" podUID="9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" containerName="registry" containerID="cri-o://f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971" gracePeriod=30
Jul 28 19:36:55.306958 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.306902    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:55.309567 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.309546    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:36:55.352992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.352922    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-installation-pull-secrets\") pod \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") "
Jul 28 19:36:55.352992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.352971    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") pod \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") "
Jul 28 19:36:55.352992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.352990    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-bound-sa-token\") pod \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") "
Jul 28 19:36:55.353217 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.353009    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-certificates\") pod \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") "
Jul 28 19:36:55.353217 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.353127    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jn6c8\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-kube-api-access-jn6c8\") pod \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") "
Jul 28 19:36:55.353217 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.353195    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-image-registry-private-configuration\") pod \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") "
Jul 28 19:36:55.353400 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.353241    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-trusted-ca\") pod \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") "
Jul 28 19:36:55.353400 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.353270    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-ca-trust-extracted\") pod \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\" (UID: \"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c\") "
Jul 28 19:36:55.353497 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.353442    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:55.353572 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.353545    2568 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-certificates\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:55.353873 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.353843    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 19:36:55.355703 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.355657    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:36:55.355819 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.355742    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:36:55.355879 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.355827    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:36:55.356128 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.356096    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:36:55.356236 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.356213    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-kube-api-access-jn6c8" (OuterVolumeSpecName: "kube-api-access-jn6c8") pod "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c"). InnerVolumeSpecName "kube-api-access-jn6c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:36:55.362211 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.362182    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" (UID: "9c14f47a-1bfb-4c6c-94a2-46f6acfd364c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:36:55.410709 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.410682    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:55.454922 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.454897    2568 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-trusted-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:55.454922 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.454918    2568 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-ca-trust-extracted\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:55.455063 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.454928    2568 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-installation-pull-secrets\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:55.455063 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.454939    2568 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-registry-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:55.455063 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.454947    2568 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-bound-sa-token\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:55.455063 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.454954    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jn6c8\" (UniqueName: \"kubernetes.io/projected/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-kube-api-access-jn6c8\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:55.455063 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.454964    2568 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c-image-registry-private-configuration\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:36:55.548055 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.548023    2568 generic.go:356] "Generic (PLEG): container finished" podID="9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" containerID="f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971" exitCode=0
Jul 28 19:36:55.548246 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.548068    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv"
Jul 28 19:36:55.548246 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.548108    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv" event={"ID":"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c","Type":"ContainerDied","Data":"f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971"}
Jul 28 19:36:55.548246 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.548166    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6db6c45f8b-r74tv" event={"ID":"9c14f47a-1bfb-4c6c-94a2-46f6acfd364c","Type":"ContainerDied","Data":"650ca0b8c365f80d9e92de5c57066a46f430e94aa8651f3f31bec6b39782e165"}
Jul 28 19:36:55.548246 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.548196    2568 scope.go:117] "RemoveContainer" containerID="f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971"
Jul 28 19:36:55.559904 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.559876    2568 scope.go:117] "RemoveContainer" containerID="f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971"
Jul 28 19:36:55.560546 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:36:55.560503    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971\": container with ID starting with f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971 not found: ID does not exist" containerID="f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971"
Jul 28 19:36:55.560669 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.560545    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971"} err="failed to get container status \"f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971\": rpc error: code = NotFound desc = could not find container \"f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971\": container with ID starting with f05c33a3b73dd54383cea5eb580da602d1d6d6276d0517b8f8ab1f317e7a4971 not found: ID does not exist"
Jul 28 19:36:55.577166 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.577128    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6db6c45f8b-r74tv"]
Jul 28 19:36:55.580367 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.580340    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6db6c45f8b-r74tv"]
Jul 28 19:36:55.616364 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.616263    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:55.675053 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.675021    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" path="/var/lib/kubelet/pods/9c14f47a-1bfb-4c6c-94a2-46f6acfd364c/volumes"
Jul 28 19:36:55.680601 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:55.680578    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:36:58.566186 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:58.566148    2568 generic.go:356] "Generic (PLEG): container finished" podID="06a4de9b-a9f5-4ff3-be14-6c8d834da836" containerID="e8cb83f4a7c2a28aeacaf10ee829cf4f0a716d8c01e59a9f643cd243a614e63c" exitCode=0
Jul 28 19:36:58.566587 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:58.566200    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz" event={"ID":"06a4de9b-a9f5-4ff3-be14-6c8d834da836","Type":"ContainerDied","Data":"e8cb83f4a7c2a28aeacaf10ee829cf4f0a716d8c01e59a9f643cd243a614e63c"}
Jul 28 19:36:58.566633 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:58.566620    2568 scope.go:117] "RemoveContainer" containerID="e8cb83f4a7c2a28aeacaf10ee829cf4f0a716d8c01e59a9f643cd243a614e63c"
Jul 28 19:36:59.572426 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:36:59.572390    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5d6c75f84f-d48hz" event={"ID":"06a4de9b-a9f5-4ff3-be14-6c8d834da836","Type":"ContainerStarted","Data":"8bcba1ef0ec0addd0c10173f9b4e48d6365a579a46af9f1faccc7e392ae18191"}
Jul 28 19:37:02.243579 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:02.243542    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:02.304465 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:02.304434    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:08.617666 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:08.617633    2568 generic.go:356] "Generic (PLEG): container finished" podID="95d94912-66ec-4945-a6ba-4c551736c24e" containerID="e71dc22db4e5b5acf3bce9523929382d6188653ba8d5a2992110e7e567c46ce1" exitCode=0
Jul 28 19:37:08.618081 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:08.617678    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-6qp7r" event={"ID":"95d94912-66ec-4945-a6ba-4c551736c24e","Type":"ContainerDied","Data":"e71dc22db4e5b5acf3bce9523929382d6188653ba8d5a2992110e7e567c46ce1"}
Jul 28 19:37:08.618081 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:08.617970    2568 scope.go:117] "RemoveContainer" containerID="e71dc22db4e5b5acf3bce9523929382d6188653ba8d5a2992110e7e567c46ce1"
Jul 28 19:37:09.623927 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:09.623884    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-5885fdb759-6qp7r" event={"ID":"95d94912-66ec-4945-a6ba-4c551736c24e","Type":"ContainerStarted","Data":"1861637dd25642fb330b74c39e29dfac37f8069124013d06049da6c7eeb5e879"}
Jul 28 19:37:49.768473 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:49.768434    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:49.826649 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:49.826615    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:50.096621 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:50.096536    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:50.189605 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:50.189565    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:50.251544 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:50.251500    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:50.797459 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:50.797420    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:37:50.899682 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:37:50.899641    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:00.544518 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:00.544477    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:00.643834 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:00.643789    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:00.706097 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:00.706063    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:01.716360 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:01.716307    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:01.817550 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:01.817515    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:17.261710 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:17.261655    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:17.357647 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:17.357614    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:27.828894 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:27.828855    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:39.575215 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:39.575182    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:39.642718 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:39.642673    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:38:44.340814 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:38:44.340774    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:03.536443 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:03.536405    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:14.783919 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:14.783878    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:14.873183 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:14.873150    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:36.530220 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:36.530187    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:37.500737 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:37.500698    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:40.194158 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:40.194123    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:40.300279 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:40.300243    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:41.818892 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:41.818855    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:41.926078 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:41.926039    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:42.204782 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:42.204747    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:42.359776 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:42.359741    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:43.842685 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:43.842654    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:44.230593 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:44.230557    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:46.959740 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:46.959708    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:47.024763 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:47.024730    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:47.515386 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:47.515352    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:39:47.522493 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:47.522466    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:39:47.526968 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:47.526943    2568 kubelet.go:1648] "Image garbage collection succeeded"
Jul 28 19:39:49.478106 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:49.478071    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:49.580671 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:49.580633    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:52.647537 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:52.647505    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:53.549362 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:53.549301    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:53.649859 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:53.649826    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:53.902392 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:53.902357    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:39:53.958751 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:39:53.958716    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:02.566719 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:02.566673    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:02.661358 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:02.661312    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:04.540738 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:04.540702    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:22.458196 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:22.458162    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:23.423530 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.423494    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"]
Jul 28 19:40:23.423802 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.423789    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" containerName="registry"
Jul 28 19:40:23.423848 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.423803    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" containerName="registry"
Jul 28 19:40:23.423848 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.423822    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0d96890-64f2-4bd4-b378-a3d79db95fec" containerName="console"
Jul 28 19:40:23.423848 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.423827    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0d96890-64f2-4bd4-b378-a3d79db95fec" containerName="console"
Jul 28 19:40:23.424242 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.423881    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0d96890-64f2-4bd4-b378-a3d79db95fec" containerName="console"
Jul 28 19:40:23.424242 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.423890    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="9c14f47a-1bfb-4c6c-94a2-46f6acfd364c" containerName="registry"
Jul 28 19:40:23.426958 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.426938    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.429773 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.429753    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 28 19:40:23.430133 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.430110    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 28 19:40:23.430189 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.430158    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 28 19:40:23.430453 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.430435    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 28 19:40:23.432030 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.432015    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k8nwg\""
Jul 28 19:40:23.450341 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.450303    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"]
Jul 28 19:40:23.543416 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.543377    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wv2ns\" (UniqueName: \"kubernetes.io/projected/9d4e5600-2f7a-43e4-8288-f2337cd1a875-kube-api-access-wv2ns\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.543416 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.543420    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d4e5600-2f7a-43e4-8288-f2337cd1a875-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.543909 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.543502    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d4e5600-2f7a-43e4-8288-f2337cd1a875-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.543909 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.543560    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d4e5600-2f7a-43e4-8288-f2337cd1a875-manifests\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.543909 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.543608    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d4e5600-2f7a-43e4-8288-f2337cd1a875-tmp\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.644278 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.644241    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wv2ns\" (UniqueName: \"kubernetes.io/projected/9d4e5600-2f7a-43e4-8288-f2337cd1a875-kube-api-access-wv2ns\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.644278 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.644283    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d4e5600-2f7a-43e4-8288-f2337cd1a875-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.644536 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.644315    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d4e5600-2f7a-43e4-8288-f2337cd1a875-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.644536 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.644354    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d4e5600-2f7a-43e4-8288-f2337cd1a875-manifests\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.644536 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.644376    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d4e5600-2f7a-43e4-8288-f2337cd1a875-tmp\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.644857 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.644829    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/9d4e5600-2f7a-43e4-8288-f2337cd1a875-manifests\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.646637 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.646608    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/9d4e5600-2f7a-43e4-8288-f2337cd1a875-tmp\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.646830 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.646813    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9d4e5600-2f7a-43e4-8288-f2337cd1a875-webhook-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.646992 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.646954    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/9d4e5600-2f7a-43e4-8288-f2337cd1a875-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.652688 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.652671    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wv2ns\" (UniqueName: \"kubernetes.io/projected/9d4e5600-2f7a-43e4-8288-f2337cd1a875-kube-api-access-wv2ns\") pod \"opendatahub-operator-controller-manager-5f4d69945-5694p\" (UID: \"9d4e5600-2f7a-43e4-8288-f2337cd1a875\") " pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.737277 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.737190    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:23.744282 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.744252    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:23.844822 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.844793    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:23.872521 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.872483    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"]
Jul 28 19:40:23.881852 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:40:23.881827    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d4e5600_2f7a_43e4_8288_f2337cd1a875.slice/crio-fd086a3b89fa839f65bb9a5e9508725bd72b69cf4b96b227b3050b5ead850bab WatchSource:0}: Error finding container fd086a3b89fa839f65bb9a5e9508725bd72b69cf4b96b227b3050b5ead850bab: Status 404 returned error can't find the container with id fd086a3b89fa839f65bb9a5e9508725bd72b69cf4b96b227b3050b5ead850bab
Jul 28 19:40:23.883827 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.883807    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:40:23.944591 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:23.944536    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:24.051453 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:24.051366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:24.384671 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:24.384584    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p" event={"ID":"9d4e5600-2f7a-43e4-8288-f2337cd1a875","Type":"ContainerStarted","Data":"fd086a3b89fa839f65bb9a5e9508725bd72b69cf4b96b227b3050b5ead850bab"}
Jul 28 19:40:27.401143 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:27.401108    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p" event={"ID":"9d4e5600-2f7a-43e4-8288-f2337cd1a875","Type":"ContainerStarted","Data":"27bcfb12fb7897f4fc083db03b9e1ca702fbaaa978efff15a7c5ad24fae58cf4"}
Jul 28 19:40:28.407008 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:28.406966    2568 generic.go:356] "Generic (PLEG): container finished" podID="9d4e5600-2f7a-43e4-8288-f2337cd1a875" containerID="27bcfb12fb7897f4fc083db03b9e1ca702fbaaa978efff15a7c5ad24fae58cf4" exitCode=0
Jul 28 19:40:28.407419 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:28.407060    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p" event={"ID":"9d4e5600-2f7a-43e4-8288-f2337cd1a875","Type":"ContainerDied","Data":"27bcfb12fb7897f4fc083db03b9e1ca702fbaaa978efff15a7c5ad24fae58cf4"}
Jul 28 19:40:28.766887 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:28.766797    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:28.845849 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:28.845817    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:29.413157 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:29.413119    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p" event={"ID":"9d4e5600-2f7a-43e4-8288-f2337cd1a875","Type":"ContainerStarted","Data":"6f22d7b26e4aac471eff0420544ca760be0fe103c7b3b3903267f179c66a54ef"}
Jul 28 19:40:29.413568 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:29.413214    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:29.449770 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:29.449717    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p" podStartSLOduration=1.986867333 podStartE2EDuration="6.449700853s" totalImagesPullingTime="3.440983864s" totalInitContainerRuntime="1.021849656s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:23 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:29.448393641 +0000 UTC m=+342.259955524" watchObservedRunningTime="2026-07-28 19:40:29.449700853 +0000 UTC m=+342.261262715"
Jul 28 19:40:40.419813 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:40.419732    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5f4d69945-5694p"
Jul 28 19:40:42.145265 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:42.145224    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:42.315067 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:42.315029    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:42.638490 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:42.638453    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:42.941801 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:42.941708    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:43.031667 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:43.031613    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:43.132733 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:43.132700    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:43.216722 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:43.216638    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:43.298687 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:43.298656    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:43.902360 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:43.902302    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:43.998653 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:43.998616    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:44.865863 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:44.865830    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:44.969572 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:44.969527    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:49.860380 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:49.860317    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:49.972017 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:49.971977    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:50.602311 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:50.602274    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:50.874884 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:50.874784    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:50.970589 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:50.970552    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:52.381551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.381515    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"]
Jul 28 19:40:52.386468 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.386443    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.390044 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.390009    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Jul 28 19:40:52.390187 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.390085    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Jul 28 19:40:52.390187 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.390108    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Jul 28 19:40:52.390302 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.390217    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Jul 28 19:40:52.390378 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.390313    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-cljhz\""
Jul 28 19:40:52.390827 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.390807    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Jul 28 19:40:52.391054 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.391031    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"]
Jul 28 19:40:52.490672 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.490638    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgj7n\" (UniqueName: \"kubernetes.io/projected/340d826d-108e-41f7-96ab-c3facd568481-kube-api-access-pgj7n\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.490902 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.490691    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/340d826d-108e-41f7-96ab-c3facd568481-manager-config\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.490902 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.490797    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/340d826d-108e-41f7-96ab-c3facd568481-cert\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.490902 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.490883    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/340d826d-108e-41f7-96ab-c3facd568481-metrics-cert\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.591994 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.591959    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pgj7n\" (UniqueName: \"kubernetes.io/projected/340d826d-108e-41f7-96ab-c3facd568481-kube-api-access-pgj7n\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.592204 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.592008    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/340d826d-108e-41f7-96ab-c3facd568481-manager-config\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.592204 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.592040    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/340d826d-108e-41f7-96ab-c3facd568481-cert\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.592204 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.592074    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/340d826d-108e-41f7-96ab-c3facd568481-metrics-cert\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.592737 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.592713    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/340d826d-108e-41f7-96ab-c3facd568481-manager-config\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.594538 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.594517    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/340d826d-108e-41f7-96ab-c3facd568481-cert\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.594634 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.594617    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/340d826d-108e-41f7-96ab-c3facd568481-metrics-cert\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.600747 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.600719    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgj7n\" (UniqueName: \"kubernetes.io/projected/340d826d-108e-41f7-96ab-c3facd568481-kube-api-access-pgj7n\") pod \"lws-controller-manager-69b7dc5fd4-w9sn8\" (UID: \"340d826d-108e-41f7-96ab-c3facd568481\") " pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.696500 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.696465    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:52.857585 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:52.857554    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"]
Jul 28 19:40:52.861354 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:40:52.861302    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod340d826d_108e_41f7_96ab_c3facd568481.slice/crio-b03b3130940d2979648f034d09cec01c52f01f800f3e1bb5b02efcdf1e9e5a90 WatchSource:0}: Error finding container b03b3130940d2979648f034d09cec01c52f01f800f3e1bb5b02efcdf1e9e5a90: Status 404 returned error can't find the container with id b03b3130940d2979648f034d09cec01c52f01f800f3e1bb5b02efcdf1e9e5a90
Jul 28 19:40:53.518970 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:53.518932    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8" event={"ID":"340d826d-108e-41f7-96ab-c3facd568481","Type":"ContainerStarted","Data":"b03b3130940d2979648f034d09cec01c52f01f800f3e1bb5b02efcdf1e9e5a90"}
Jul 28 19:40:54.475945 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.475781    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:54.546122 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.546085    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"]
Jul 28 19:40:54.549562 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.549539    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:54.553120 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.553091    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-jnqft\""
Jul 28 19:40:54.553256 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.553096    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 28 19:40:54.557309 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.557239    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"]
Jul 28 19:40:54.560706 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.560687    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:40:54.564675 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.564216    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-v26nq\""
Jul 28 19:40:54.565722 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.565363    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 28 19:40:54.566066 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.566041    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"]
Jul 28 19:40:54.579383 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.579356    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"]
Jul 28 19:40:54.605647 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.605607    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1089c743-484f-4e6f-b82b-2cf6249a9b43-cert\") pod \"kserve-controller-manager-d6f4b65b8-l8vf8\" (UID: \"1089c743-484f-4e6f-b82b-2cf6249a9b43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:54.605830 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.605658    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/212d3cb5-ad7a-463a-9ab3-7bf10f94afaa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mwzqd\" (UID: \"212d3cb5-ad7a-463a-9ab3-7bf10f94afaa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:40:54.605830 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.605746    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72scb\" (UniqueName: \"kubernetes.io/projected/212d3cb5-ad7a-463a-9ab3-7bf10f94afaa-kube-api-access-72scb\") pod \"llmisvc-controller-manager-777c6bd7d5-mwzqd\" (UID: \"212d3cb5-ad7a-463a-9ab3-7bf10f94afaa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:40:54.605830 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.605789    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-98p5b\" (UniqueName: \"kubernetes.io/projected/1089c743-484f-4e6f-b82b-2cf6249a9b43-kube-api-access-98p5b\") pod \"kserve-controller-manager-d6f4b65b8-l8vf8\" (UID: \"1089c743-484f-4e6f-b82b-2cf6249a9b43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:54.607406 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.607364    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:54.706646 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.706302    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1089c743-484f-4e6f-b82b-2cf6249a9b43-cert\") pod \"kserve-controller-manager-d6f4b65b8-l8vf8\" (UID: \"1089c743-484f-4e6f-b82b-2cf6249a9b43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:54.706646 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.706373    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/212d3cb5-ad7a-463a-9ab3-7bf10f94afaa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mwzqd\" (UID: \"212d3cb5-ad7a-463a-9ab3-7bf10f94afaa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:40:54.706646 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.706421    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72scb\" (UniqueName: \"kubernetes.io/projected/212d3cb5-ad7a-463a-9ab3-7bf10f94afaa-kube-api-access-72scb\") pod \"llmisvc-controller-manager-777c6bd7d5-mwzqd\" (UID: \"212d3cb5-ad7a-463a-9ab3-7bf10f94afaa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:40:54.706646 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.706424    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:54.706646 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.706456    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-98p5b\" (UniqueName: \"kubernetes.io/projected/1089c743-484f-4e6f-b82b-2cf6249a9b43-kube-api-access-98p5b\") pod \"kserve-controller-manager-d6f4b65b8-l8vf8\" (UID: \"1089c743-484f-4e6f-b82b-2cf6249a9b43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:54.709111 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.709083    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1089c743-484f-4e6f-b82b-2cf6249a9b43-cert\") pod \"kserve-controller-manager-d6f4b65b8-l8vf8\" (UID: \"1089c743-484f-4e6f-b82b-2cf6249a9b43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:54.709217 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.709196    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/212d3cb5-ad7a-463a-9ab3-7bf10f94afaa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-mwzqd\" (UID: \"212d3cb5-ad7a-463a-9ab3-7bf10f94afaa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:40:54.716714 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.716695    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-98p5b\" (UniqueName: \"kubernetes.io/projected/1089c743-484f-4e6f-b82b-2cf6249a9b43-kube-api-access-98p5b\") pod \"kserve-controller-manager-d6f4b65b8-l8vf8\" (UID: \"1089c743-484f-4e6f-b82b-2cf6249a9b43\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:54.716988 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.716942    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72scb\" (UniqueName: \"kubernetes.io/projected/212d3cb5-ad7a-463a-9ab3-7bf10f94afaa-kube-api-access-72scb\") pod \"llmisvc-controller-manager-777c6bd7d5-mwzqd\" (UID: \"212d3cb5-ad7a-463a-9ab3-7bf10f94afaa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:40:54.832395 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.832270    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:54.865780 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.865731    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:54.882414 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:54.882386    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:40:55.009110 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:55.009065    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:40:55.991503 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:55.991428    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"]
Jul 28 19:40:55.994052 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:40:55.993994    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod212d3cb5_ad7a_463a_9ab3_7bf10f94afaa.slice/crio-d0c5b59fa6d70860513744b06e15091245b38b11e5b3ada34ac58009820ea93c WatchSource:0}: Error finding container d0c5b59fa6d70860513744b06e15091245b38b11e5b3ada34ac58009820ea93c: Status 404 returned error can't find the container with id d0c5b59fa6d70860513744b06e15091245b38b11e5b3ada34ac58009820ea93c
Jul 28 19:40:56.011491 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:56.011461    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"]
Jul 28 19:40:56.014898 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:40:56.014860    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1089c743_484f_4e6f_b82b_2cf6249a9b43.slice/crio-cf5818025bd0d62bdd22d29787de7fac5ef3710dfe1104ad64b7f1966af969e7 WatchSource:0}: Error finding container cf5818025bd0d62bdd22d29787de7fac5ef3710dfe1104ad64b7f1966af969e7: Status 404 returned error can't find the container with id cf5818025bd0d62bdd22d29787de7fac5ef3710dfe1104ad64b7f1966af969e7
Jul 28 19:40:56.534065 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:56.534026    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8" event={"ID":"1089c743-484f-4e6f-b82b-2cf6249a9b43","Type":"ContainerStarted","Data":"cf5818025bd0d62bdd22d29787de7fac5ef3710dfe1104ad64b7f1966af969e7"}
Jul 28 19:40:56.535428 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:56.535393    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8" event={"ID":"340d826d-108e-41f7-96ab-c3facd568481","Type":"ContainerStarted","Data":"1d5c0c1cf3bef5bf157c07927bff57d71826156317247b610cc2165bc45b3bc5"}
Jul 28 19:40:56.535556 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:56.535495    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:40:56.536598 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:56.536579    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd" event={"ID":"212d3cb5-ad7a-463a-9ab3-7bf10f94afaa","Type":"ContainerStarted","Data":"d0c5b59fa6d70860513744b06e15091245b38b11e5b3ada34ac58009820ea93c"}
Jul 28 19:40:56.575705 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:56.575652    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8" podStartSLOduration=1.520590476 podStartE2EDuration="4.575626756s" totalImagesPullingTime="3.05503628s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:56.5746699 +0000 UTC m=+369.386231764" watchObservedRunningTime="2026-07-28 19:40:56.575626756 +0000 UTC m=+369.387188618"
Jul 28 19:40:58.548600 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:58.548564    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8" event={"ID":"1089c743-484f-4e6f-b82b-2cf6249a9b43","Type":"ContainerStarted","Data":"f45ac0eb067ae79861958137a13f5b2feca9cc5c6296d9b69a181e1d96d98a7d"}
Jul 28 19:40:58.549031 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:58.548634    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:40:58.564892 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:58.564688    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8" podStartSLOduration=2.161047039 podStartE2EDuration="4.564669241s" totalImagesPullingTime="2.403622202s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:58.564290839 +0000 UTC m=+371.375852741" watchObservedRunningTime="2026-07-28 19:40:58.564669241 +0000 UTC m=+371.376231106"
Jul 28 19:40:59.554467 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:59.554427    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd" event={"ID":"212d3cb5-ad7a-463a-9ab3-7bf10f94afaa","Type":"ContainerStarted","Data":"4a2794abe5b43f02806817e5dc2bd84e44eb1bbc3686013b3651bb2e35a29a7e"}
Jul 28 19:40:59.572112 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:40:59.572057    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd" podStartSLOduration=2.517774971 podStartE2EDuration="5.572040794s" totalImagesPullingTime="3.054265823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:40:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:40:59.570718976 +0000 UTC m=+372.382280859" watchObservedRunningTime="2026-07-28 19:40:59.572040794 +0000 UTC m=+372.383602654"
Jul 28 19:41:00.558697 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:00.558666    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:41:07.544986 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:07.544953    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-69b7dc5fd4-w9sn8"
Jul 28 19:41:12.252073 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:12.252036    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:29.560171 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:29.560137    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-l8vf8"
Jul 28 19:41:31.565034 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:31.565000    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-mwzqd"
Jul 28 19:41:43.591300 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:43.591258    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:43.697181 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:43.697133    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:43.935188 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:43.935141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:54.683695 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.683659    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c8dfc9b74-pjltb"]
Jul 28 19:41:54.687614 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.687585    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.691885 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.691857    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Jul 28 19:41:54.692060 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.691858    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Jul 28 19:41:54.692473 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.692444    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Jul 28 19:41:54.693145 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.693074    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Jul 28 19:41:54.693145 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.693098    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Jul 28 19:41:54.693296 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.693228    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dkbk4\""
Jul 28 19:41:54.698106 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.698079    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Jul 28 19:41:54.699626 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.699606    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8dfc9b74-pjltb"]
Jul 28 19:41:54.754176 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.754137    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:54.792525 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.792492    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-config\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.792525 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.792525    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xq8km\" (UniqueName: \"kubernetes.io/projected/e50233f5-80d6-4577-91d3-b1e8b3f5189e-kube-api-access-xq8km\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.792729 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.792637    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-service-ca\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.792729 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.792654    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-oauth-serving-cert\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.792729 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.792676    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-trusted-ca-bundle\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.792729 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.792695    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-oauth-config\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.792868 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.792802    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-serving-cert\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.828765 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.828734    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:54.893550 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.893513    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-service-ca\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.893550 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.893551    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-oauth-serving-cert\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.893799 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.893582    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-trusted-ca-bundle\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.893799 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.893702    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-oauth-config\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.893799 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.893773    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-serving-cert\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.893949 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.893839    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-config\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.893949 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.893867    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xq8km\" (UniqueName: \"kubernetes.io/projected/e50233f5-80d6-4577-91d3-b1e8b3f5189e-kube-api-access-xq8km\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.894810 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.894783    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-trusted-ca-bundle\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.894993 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.894732    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-service-ca\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.895099 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.895051    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-config\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.895364 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.895337    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e50233f5-80d6-4577-91d3-b1e8b3f5189e-oauth-serving-cert\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.900317 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.900291    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-oauth-config\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.900473 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.900450    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e50233f5-80d6-4577-91d3-b1e8b3f5189e-console-serving-cert\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.904284 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.904262    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xq8km\" (UniqueName: \"kubernetes.io/projected/e50233f5-80d6-4577-91d3-b1e8b3f5189e-kube-api-access-xq8km\") pod \"console-c8dfc9b74-pjltb\" (UID: \"e50233f5-80d6-4577-91d3-b1e8b3f5189e\") " pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:54.929218 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:54.929185    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:55.001654 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:55.001564    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:41:55.142967 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:55.142838    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c8dfc9b74-pjltb"]
Jul 28 19:41:55.145215 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:41:55.145181    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode50233f5_80d6_4577_91d3_b1e8b3f5189e.slice/crio-2275a9c74df81bcfc0c45026e89f582cda02ce36f47ae3737dfeb39dd2daea2d WatchSource:0}: Error finding container 2275a9c74df81bcfc0c45026e89f582cda02ce36f47ae3737dfeb39dd2daea2d: Status 404 returned error can't find the container with id 2275a9c74df81bcfc0c45026e89f582cda02ce36f47ae3737dfeb39dd2daea2d
Jul 28 19:41:55.486137 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:55.486105    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:55.554730 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:55.554685    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:41:55.796464 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:55.796373    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8dfc9b74-pjltb" event={"ID":"e50233f5-80d6-4577-91d3-b1e8b3f5189e","Type":"ContainerStarted","Data":"a54e4ff9b410e734fe751abddbb03139704c6b5a3c2e0888546b61f11f6dce3f"}
Jul 28 19:41:55.796464 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:55.796427    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c8dfc9b74-pjltb" event={"ID":"e50233f5-80d6-4577-91d3-b1e8b3f5189e","Type":"ContainerStarted","Data":"2275a9c74df81bcfc0c45026e89f582cda02ce36f47ae3737dfeb39dd2daea2d"}
Jul 28 19:41:55.826244 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:41:55.826189    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c8dfc9b74-pjltb" podStartSLOduration=1.826173466 podStartE2EDuration="1.826173466s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:41:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:41:55.824774996 +0000 UTC m=+428.636336859" watchObservedRunningTime="2026-07-28 19:41:55.826173466 +0000 UTC m=+428.637735328"
Jul 28 19:42:05.002136 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:05.002100    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:42:05.002618 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:05.002157    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:42:05.006865 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:05.006840    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:42:05.843595 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:05.843566    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c8dfc9b74-pjltb"
Jul 28 19:42:06.018796 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:06.018749    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:06.118763 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:06.118679    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:06.218695 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:06.218661    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:12.803117 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:12.803075    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:16.407497 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:16.407450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:19.478199 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:19.478162    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:20.919931 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:20.919893    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:21.338659 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:21.338555    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:21.806895 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:21.806845    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:24.455765 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:24.455728    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:24.560340 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:24.560296    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:25.003241 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:25.003204    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:27.074565 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:27.074534    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:28.666081 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:28.666043    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:28.772261 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:28.772221    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:31.089885 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.089850    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"]
Jul 28 19:42:31.093747 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.093726    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.096644 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.096623    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 28 19:42:31.096987 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.096947    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Jul 28 19:42:31.097251 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.097221    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6gsrx\""
Jul 28 19:42:31.097338 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.097227    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Jul 28 19:42:31.098074 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.098059    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 28 19:42:31.103638 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.103612    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"]
Jul 28 19:42:31.111601 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.111562    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:31.190623 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.190580    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/613c075a-25d1-4ba6-a35c-bceafb4a65cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.190623 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.190619    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwf2k\" (UniqueName: \"kubernetes.io/projected/613c075a-25d1-4ba6-a35c-bceafb4a65cf-kube-api-access-jwf2k\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.190869 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.190711    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/613c075a-25d1-4ba6-a35c-bceafb4a65cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.291824 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.291788    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/613c075a-25d1-4ba6-a35c-bceafb4a65cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.291824 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.291830    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwf2k\" (UniqueName: \"kubernetes.io/projected/613c075a-25d1-4ba6-a35c-bceafb4a65cf-kube-api-access-jwf2k\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.292073 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:42:31.291980    2568 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found
Jul 28 19:42:31.292131 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:42:31.292074    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/613c075a-25d1-4ba6-a35c-bceafb4a65cf-plugin-serving-cert podName:613c075a-25d1-4ba6-a35c-bceafb4a65cf nodeName:}" failed. No retries permitted until 2026-07-28 19:42:31.792050946 +0000 UTC m=+464.603612802 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/613c075a-25d1-4ba6-a35c-bceafb4a65cf-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-p2kx2" (UID: "613c075a-25d1-4ba6-a35c-bceafb4a65cf") : secret "plugin-serving-cert" not found
Jul 28 19:42:31.292131 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.292100    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/613c075a-25d1-4ba6-a35c-bceafb4a65cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.292816 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.292795    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/613c075a-25d1-4ba6-a35c-bceafb4a65cf-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.304137 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.304109    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwf2k\" (UniqueName: \"kubernetes.io/projected/613c075a-25d1-4ba6-a35c-bceafb4a65cf-kube-api-access-jwf2k\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.615936 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.615899    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:31.796722 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.796676    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/613c075a-25d1-4ba6-a35c-bceafb4a65cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:31.799113 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:31.799083    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/613c075a-25d1-4ba6-a35c-bceafb4a65cf-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-p2kx2\" (UID: \"613c075a-25d1-4ba6-a35c-bceafb4a65cf\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:32.004382 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:32.004314    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"
Jul 28 19:42:32.135922 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:32.135760    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2"]
Jul 28 19:42:32.138258 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:42:32.138233    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod613c075a_25d1_4ba6_a35c_bceafb4a65cf.slice/crio-d06e153806f4f072a3eb918f4afdde97b0b419033ee526a401da380af7f3fe90 WatchSource:0}: Error finding container d06e153806f4f072a3eb918f4afdde97b0b419033ee526a401da380af7f3fe90: Status 404 returned error can't find the container with id d06e153806f4f072a3eb918f4afdde97b0b419033ee526a401da380af7f3fe90
Jul 28 19:42:32.424840 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:32.424802    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:32.524466 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:32.524427    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:32.953553 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:32.953521    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2" event={"ID":"613c075a-25d1-4ba6-a35c-bceafb4a65cf","Type":"ContainerStarted","Data":"d06e153806f4f072a3eb918f4afdde97b0b419033ee526a401da380af7f3fe90"}
Jul 28 19:42:41.989543 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:41.989501    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:42.219406 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.219363    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:42.335317 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.335234    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"]
Jul 28 19:42:42.356632 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.356603    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:42.360955 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.360930    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-lcnz8\""
Jul 28 19:42:42.375993 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.375959    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"]
Jul 28 19:42:42.426841 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.426801    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:42.496508 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.496468    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fec845ed-6569-41a3-8a07-f67ca2b9e659-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-njfjs\" (UID: \"fec845ed-6569-41a3-8a07-f67ca2b9e659\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:42.496683 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.496585    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlt44\" (UniqueName: \"kubernetes.io/projected/fec845ed-6569-41a3-8a07-f67ca2b9e659-kube-api-access-rlt44\") pod \"kuadrant-operator-controller-manager-86878c769f-njfjs\" (UID: \"fec845ed-6569-41a3-8a07-f67ca2b9e659\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:42.597532 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.597441    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fec845ed-6569-41a3-8a07-f67ca2b9e659-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-njfjs\" (UID: \"fec845ed-6569-41a3-8a07-f67ca2b9e659\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:42.597697 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.597555    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlt44\" (UniqueName: \"kubernetes.io/projected/fec845ed-6569-41a3-8a07-f67ca2b9e659-kube-api-access-rlt44\") pod \"kuadrant-operator-controller-manager-86878c769f-njfjs\" (UID: \"fec845ed-6569-41a3-8a07-f67ca2b9e659\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:42.597880 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.597858    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fec845ed-6569-41a3-8a07-f67ca2b9e659-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-njfjs\" (UID: \"fec845ed-6569-41a3-8a07-f67ca2b9e659\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:42.618455 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.618428    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlt44\" (UniqueName: \"kubernetes.io/projected/fec845ed-6569-41a3-8a07-f67ca2b9e659-kube-api-access-rlt44\") pod \"kuadrant-operator-controller-manager-86878c769f-njfjs\" (UID: \"fec845ed-6569-41a3-8a07-f67ca2b9e659\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:42.669875 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:42.669839    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:43.175748 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:43.175711    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:44.956508 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:44.956464    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:45.057308 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:45.057269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:47.537578 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:47.537535    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:47.633279 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:47.633240    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:51.238089 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:51.238046    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:51.353131 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:51.353088    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:54.202850 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:54.202825    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"]
Jul 28 19:42:54.204618 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:42:54.204591    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfec845ed_6569_41a3_8a07_f67ca2b9e659.slice/crio-5f223f0f9810b1d22e4cd3e731d2a452be2eb5d0a52a28c5a81e770b4574a270 WatchSource:0}: Error finding container 5f223f0f9810b1d22e4cd3e731d2a452be2eb5d0a52a28c5a81e770b4574a270: Status 404 returned error can't find the container with id 5f223f0f9810b1d22e4cd3e731d2a452be2eb5d0a52a28c5a81e770b4574a270
Jul 28 19:42:55.031600 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:55.031546    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:55.072246 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:55.072184    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2" event={"ID":"613c075a-25d1-4ba6-a35c-bceafb4a65cf","Type":"ContainerStarted","Data":"191418f0a433f7f0e8a5762fba15def1da25d3b10abe17cd41e5ef961e50b3e1"}
Jul 28 19:42:55.073853 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:55.073811    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs" event={"ID":"fec845ed-6569-41a3-8a07-f67ca2b9e659","Type":"ContainerStarted","Data":"5f223f0f9810b1d22e4cd3e731d2a452be2eb5d0a52a28c5a81e770b4574a270"}
Jul 28 19:42:55.093027 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:55.092964    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-p2kx2" podStartSLOduration=2.129617148 podStartE2EDuration="24.092946691s" totalImagesPullingTime="21.963329543s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:55.089781731 +0000 UTC m=+487.901343596" watchObservedRunningTime="2026-07-28 19:42:55.092946691 +0000 UTC m=+487.904508554"
Jul 28 19:42:55.127483 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:55.127448    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:42:59.098118 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:59.098030    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs" event={"ID":"fec845ed-6569-41a3-8a07-f67ca2b9e659","Type":"ContainerStarted","Data":"11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182"}
Jul 28 19:42:59.098575 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:59.098126    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:42:59.119722 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:42:59.119672    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs" podStartSLOduration=12.534486253 podStartE2EDuration="17.11965702s" totalImagesPullingTime="4.585170767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:42:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:42:59.117104692 +0000 UTC m=+491.928666554" watchObservedRunningTime="2026-07-28 19:42:59.11965702 +0000 UTC m=+491.931218862"
Jul 28 19:43:00.026864 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:00.026823    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:00.092302 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:00.092266    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:00.219691 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:00.219651    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:00.316102 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:00.316019    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:02.829379 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:02.829340    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:02.931056 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:02.931013    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:10.105153 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:10.105121    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:43:26.588590 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:26.588537    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:27.551376 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:27.551342    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:32.000566 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:32.000531    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:32.115304 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:32.115269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:32.224960 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:32.224921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:32.790175 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:32.790141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:32.851630 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:32.851589    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:46.618845 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:46.618807    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:53.265767 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:53.265729    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:53.367570 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:53.367537    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:43:53.763965 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:43:53.763931    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:06.325980 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:06.325945    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:06.746769 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:06.746734    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:14.037287 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:14.037251    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:14.106830 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:14.106795    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:14.164836 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:14.164794    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:14.299845 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:14.299756    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:17.477139 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:17.477108    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:17.679842 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:17.679807    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:40.913845 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:40.913810    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:41.015381 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:41.015346    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:47.546938 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:47.546906    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:44:47.555397 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:47.555374    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:44:50.990839 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:50.990802    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:51.084621 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:51.084585    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:51.488822 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:51.488791    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:51.764873 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:51.764782    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:52.053758 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:52.053656    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:44:52.224943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:44:52.224907    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:03.334548 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:03.334484    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:03.433271 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:03.433236    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:10.611461 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:10.611344    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:12.101773 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:12.101736    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:17.258634 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.258599    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"]
Jul 28 19:45:17.262052 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.262031    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.266211 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.266187    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 28 19:45:17.266315 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.266213    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Jul 28 19:45:17.266315 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.266191    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-jpg4f\""
Jul 28 19:45:17.266823 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.266807    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 28 19:45:17.276518 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.276498    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"]
Jul 28 19:45:17.295297 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.295258    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:17.387613 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.387581    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.387813 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.387620    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhw8k\" (UniqueName: \"kubernetes.io/projected/6dc95503-ef27-42fd-a142-0a5122bcfcaf-kube-api-access-nhw8k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.387813 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.387663    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.387813 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.387779    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.387947 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.387835    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.387947 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.387874    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dc95503-ef27-42fd-a142-0a5122bcfcaf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489016 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.488973    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.489041    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.489075    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dc95503-ef27-42fd-a142-0a5122bcfcaf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489173 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.489148    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489346 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.489172    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhw8k\" (UniqueName: \"kubernetes.io/projected/6dc95503-ef27-42fd-a142-0a5122bcfcaf-kube-api-access-nhw8k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489346 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.489205    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489530 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.489426    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489771 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.489740    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.489887 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.489775    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.491417 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.491396    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6dc95503-ef27-42fd-a142-0a5122bcfcaf-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.492038 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.492016    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6dc95503-ef27-42fd-a142-0a5122bcfcaf-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.497405 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.497380    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhw8k\" (UniqueName: \"kubernetes.io/projected/6dc95503-ef27-42fd-a142-0a5122bcfcaf-kube-api-access-nhw8k\") pod \"e2e-distinct-simulated-kserve-59975d69f9-c96zb\" (UID: \"6dc95503-ef27-42fd-a142-0a5122bcfcaf\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.572184 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.572107    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:17.736461 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:17.736431    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"]
Jul 28 19:45:17.740004 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:45:17.739975    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc95503_ef27_42fd_a142_0a5122bcfcaf.slice/crio-32ca649f9c5e4e18bfc53837d0238cd01c5e6615854af0ebde2104715c1e0dc2 WatchSource:0}: Error finding container 32ca649f9c5e4e18bfc53837d0238cd01c5e6615854af0ebde2104715c1e0dc2: Status 404 returned error can't find the container with id 32ca649f9c5e4e18bfc53837d0238cd01c5e6615854af0ebde2104715c1e0dc2
Jul 28 19:45:18.721809 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:18.721765    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb" event={"ID":"6dc95503-ef27-42fd-a142-0a5122bcfcaf","Type":"ContainerStarted","Data":"32ca649f9c5e4e18bfc53837d0238cd01c5e6615854af0ebde2104715c1e0dc2"}
Jul 28 19:45:19.728090 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:19.728049    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb" event={"ID":"6dc95503-ef27-42fd-a142-0a5122bcfcaf","Type":"ContainerStarted","Data":"fae969fd10ed5acb2da56d37c8d3661d7d83f8e04efeace773b841b4cd7e1974"}
Jul 28 19:45:19.749522 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:19.749473    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb" podStartSLOduration=1.726964491 podStartE2EDuration="2.749456903s" totalImagesPullingTime="1.022492412s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:19.747973539 +0000 UTC m=+632.559535407" watchObservedRunningTime="2026-07-28 19:45:19.749456903 +0000 UTC m=+632.561018787"
Jul 28 19:45:27.572533 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:27.572491    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:27.573002 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:27.572559    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:27.585124 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:27.585093    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:27.774952 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:27.774923    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-c96zb"
Jul 28 19:45:42.773560 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:42.773522    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:49.442911 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.442874    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"]
Jul 28 19:45:49.446743 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.446715    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.449541 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.449521    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 28 19:45:49.457283 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.457259    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"]
Jul 28 19:45:49.474152 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.474128    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.474275 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.474169    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zkmrv\" (UniqueName: \"kubernetes.io/projected/012824d0-5672-4ba2-858d-51b42153f0c5-kube-api-access-zkmrv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.474275 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.474188    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.474275 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.474207    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.474275 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.474228    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.474449 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.474385    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/012824d0-5672-4ba2-858d-51b42153f0c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.546250 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.546216    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:49.575658 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.575625    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/012824d0-5672-4ba2-858d-51b42153f0c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.575821 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.575687    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.575821 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.575733    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zkmrv\" (UniqueName: \"kubernetes.io/projected/012824d0-5672-4ba2-858d-51b42153f0c5-kube-api-access-zkmrv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.575821 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.575761    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.575821 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.575793    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.575821 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.575816    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.576185 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.576156    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.576256 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.576184    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.576256 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.576211    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.577995 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.577976    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/012824d0-5672-4ba2-858d-51b42153f0c5-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.578105 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.578047    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/012824d0-5672-4ba2-858d-51b42153f0c5-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.583047 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.583023    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zkmrv\" (UniqueName: \"kubernetes.io/projected/012824d0-5672-4ba2-858d-51b42153f0c5-kube-api-access-zkmrv\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb\" (UID: \"012824d0-5672-4ba2-858d-51b42153f0c5\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.757182 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.757092    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:49.887979 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.887936    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"]
Jul 28 19:45:49.891472 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:45:49.891441    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod012824d0_5672_4ba2_858d_51b42153f0c5.slice/crio-45cdf28e4dd04e52493191e45d00e35efb4f667e841432a103f9cefd6861cf1c WatchSource:0}: Error finding container 45cdf28e4dd04e52493191e45d00e35efb4f667e841432a103f9cefd6861cf1c: Status 404 returned error can't find the container with id 45cdf28e4dd04e52493191e45d00e35efb4f667e841432a103f9cefd6861cf1c
Jul 28 19:45:49.893403 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:49.893382    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 19:45:50.871899 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:50.871864    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb" event={"ID":"012824d0-5672-4ba2-858d-51b42153f0c5","Type":"ContainerStarted","Data":"bf0fbc2a77c0d007de59b72842825a35ac4f7dfc4469e4182605d8d13eb267cb"}
Jul 28 19:45:50.872272 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:50.871914    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb" event={"ID":"012824d0-5672-4ba2-858d-51b42153f0c5","Type":"ContainerStarted","Data":"45cdf28e4dd04e52493191e45d00e35efb4f667e841432a103f9cefd6861cf1c"}
Jul 28 19:45:50.889838 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:50.889784    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb" podStartSLOduration=1.709016356 podStartE2EDuration="1.889764463s" totalImagesPullingTime="180.748107ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:45:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:45:50.888176306 +0000 UTC m=+663.699738168" watchObservedRunningTime="2026-07-28 19:45:50.889764463 +0000 UTC m=+663.701326329"
Jul 28 19:45:53.458856 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:53.458805    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:53.560597 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:53.560561    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:59.618020 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:59.617985    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:45:59.758100 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:59.758064    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:59.758259 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:59.758134    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:59.773497 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:59.773466    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:45:59.926175 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:45:59.926141    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb"
Jul 28 19:46:11.450003 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.449969    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"]
Jul 28 19:46:11.453499 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.453482    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.455899 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.455879    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 28 19:46:11.463576 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.463554    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"]
Jul 28 19:46:11.482984 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.482959    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:46:11.570538 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.570500    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.570737 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.570550    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.570737 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.570569    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.570737 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.570593    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.570737 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.570639    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.570918 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.570761    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc8z6\" (UniqueName: \"kubernetes.io/projected/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-kube-api-access-tc8z6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.671439 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.671397    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc8z6\" (UniqueName: \"kubernetes.io/projected/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-kube-api-access-tc8z6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.671613 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.671453    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.671613 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.671495    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.671613 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.671518    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.671613 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.671549    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.671613 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.671575    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.672006 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.671980    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.672086 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.672030    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.672086 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.672031    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.680108 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.674014    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.681294 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.681270    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.682785 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.682757    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc8z6\" (UniqueName: \"kubernetes.io/projected/a0f82583-c9b7-4004-8a08-1ff05f5e7b0f-kube-api-access-tc8z6\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5p4wc\" (UID: \"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.765010 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.764905    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:11.905833 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.905809    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"]
Jul 28 19:46:11.907983 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:46:11.907952    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0f82583_c9b7_4004_8a08_1ff05f5e7b0f.slice/crio-b87f9425ba0fc25a61dad939ce8bd15066a4ab49af213eb0bcd1407dd7d37df6 WatchSource:0}: Error finding container b87f9425ba0fc25a61dad939ce8bd15066a4ab49af213eb0bcd1407dd7d37df6: Status 404 returned error can't find the container with id b87f9425ba0fc25a61dad939ce8bd15066a4ab49af213eb0bcd1407dd7d37df6
Jul 28 19:46:11.969631 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:11.969599    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc" event={"ID":"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f","Type":"ContainerStarted","Data":"b87f9425ba0fc25a61dad939ce8bd15066a4ab49af213eb0bcd1407dd7d37df6"}
Jul 28 19:46:12.975566 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:12.975528    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc" event={"ID":"a0f82583-c9b7-4004-8a08-1ff05f5e7b0f","Type":"ContainerStarted","Data":"3ddd0274017912f1a6ddc17b61373303a83ce91d37311b08abf026daae39a1e6"}
Jul 28 19:46:12.993676 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:12.993611    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc" podStartSLOduration=1.781037628 podStartE2EDuration="1.993596094s" totalImagesPullingTime="212.558466ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:46:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:46:12.991538584 +0000 UTC m=+685.803100448" watchObservedRunningTime="2026-07-28 19:46:12.993596094 +0000 UTC m=+685.805157957"
Jul 28 19:46:17.405305 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.405269    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"]
Jul 28 19:46:17.408699 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.408679    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.412717 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.412695    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Jul 28 19:46:17.436871 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.436846    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"]
Jul 28 19:46:17.473127 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.473097    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:46:17.525873 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.525841    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f5whq\" (UniqueName: \"kubernetes.io/projected/fdde63f6-f71c-4147-a8f4-eb86c9082924-kube-api-access-f5whq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.526004 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.525887    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.526004 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.525904    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.526126 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.526027    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdde63f6-f71c-4147-a8f4-eb86c9082924-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.526126 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.526111    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.526229 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.526164    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627172 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627133    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdde63f6-f71c-4147-a8f4-eb86c9082924-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627395 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627192    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627395 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627220    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627395 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627257    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f5whq\" (UniqueName: \"kubernetes.io/projected/fdde63f6-f71c-4147-a8f4-eb86c9082924-kube-api-access-f5whq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627395 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627296    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627395 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627337    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627803 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627777    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627848 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627825    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.627895 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.627832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.630065 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.630039    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fdde63f6-f71c-4147-a8f4-eb86c9082924-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.630334 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.630302    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fdde63f6-f71c-4147-a8f4-eb86c9082924-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.636235 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.636213    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f5whq\" (UniqueName: \"kubernetes.io/projected/fdde63f6-f71c-4147-a8f4-eb86c9082924-kube-api-access-f5whq\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs\" (UID: \"fdde63f6-f71c-4147-a8f4-eb86c9082924\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.718391 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.718290    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:17.858049 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.858025    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"]
Jul 28 19:46:17.860412 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:46:17.860374    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfdde63f6_f71c_4147_a8f4_eb86c9082924.slice/crio-1afc256594aaaefaabcc29f21442c4ac7f2eecf8b91109c09ab5bd3f02e6619a WatchSource:0}: Error finding container 1afc256594aaaefaabcc29f21442c4ac7f2eecf8b91109c09ab5bd3f02e6619a: Status 404 returned error can't find the container with id 1afc256594aaaefaabcc29f21442c4ac7f2eecf8b91109c09ab5bd3f02e6619a
Jul 28 19:46:17.999629 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:17.999550    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs" event={"ID":"fdde63f6-f71c-4147-a8f4-eb86c9082924","Type":"ContainerStarted","Data":"1afc256594aaaefaabcc29f21442c4ac7f2eecf8b91109c09ab5bd3f02e6619a"}
Jul 28 19:46:19.006068 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:19.006027    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs" event={"ID":"fdde63f6-f71c-4147-a8f4-eb86c9082924","Type":"ContainerStarted","Data":"8da10b0cb9e4a654757a1cb24e8eff6045ed4cdcdc0e5cbb00e69496b6ad03fa"}
Jul 28 19:46:19.025823 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:19.025768    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs" podStartSLOduration=1.697525403 podStartE2EDuration="2.025750581s" totalImagesPullingTime="328.225178ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:46:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:46:19.023911293 +0000 UTC m=+691.835473162" watchObservedRunningTime="2026-07-28 19:46:19.025750581 +0000 UTC m=+691.837312443"
Jul 28 19:46:21.765492 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:21.765457    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:21.765950 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:21.765514    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:21.777966 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:21.777935    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:22.031689 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:22.031598    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5p4wc"
Jul 28 19:46:27.718943 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:27.718913    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:27.719416 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:27.718970    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:27.731759 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:27.731733    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:46:28.059401 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:46:28.059308    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs"
Jul 28 19:47:02.993642 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:47:02.993601    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:03.086673 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:47:03.086634    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:04.225316 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:47:04.225270    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:04.320386 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:47:04.320337    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:40.136195 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:47:40.136152    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:47:40.237419 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:47:40.237383    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:28.153307 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:48:28.153271    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:48:28.208649 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:48:28.208616    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:10.028833 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:10.028797    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:11.275619 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:11.275583    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:20.227808 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:20.227769    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:47.577693 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:47.577663    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:49:47.583295 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:47.583270    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:49:53.348145 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:53.348080    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:53.447369 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:53.447313    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:53.633550 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:53.633468    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:59.459800 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:59.459760    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:59.574625 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:59.574592    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:59.702148 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:59.702110    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:49:59.884423 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:49:59.884311    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:05.633861 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:50:05.633824    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:50:05.697768 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:50:05.697738    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:01.349017 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:01.348984    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:03.572415 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:03.572378    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:14.044228 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:14.044194    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"]
Jul 28 19:54:14.044730 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:14.044415    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs" podUID="fec845ed-6569-41a3-8a07-f67ca2b9e659" containerName="manager" containerID="cri-o://11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182" gracePeriod=10
Jul 28 19:54:16.903536 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:16.903508    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:54:16.958712 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:16.958627    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fec845ed-6569-41a3-8a07-f67ca2b9e659-extensions-socket-volume\") pod \"fec845ed-6569-41a3-8a07-f67ca2b9e659\" (UID: \"fec845ed-6569-41a3-8a07-f67ca2b9e659\") "
Jul 28 19:54:16.958854 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:16.958762    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rlt44\" (UniqueName: \"kubernetes.io/projected/fec845ed-6569-41a3-8a07-f67ca2b9e659-kube-api-access-rlt44\") pod \"fec845ed-6569-41a3-8a07-f67ca2b9e659\" (UID: \"fec845ed-6569-41a3-8a07-f67ca2b9e659\") "
Jul 28 19:54:16.959069 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:16.959047    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fec845ed-6569-41a3-8a07-f67ca2b9e659-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "fec845ed-6569-41a3-8a07-f67ca2b9e659" (UID: "fec845ed-6569-41a3-8a07-f67ca2b9e659"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:54:16.960859 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:16.960832    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fec845ed-6569-41a3-8a07-f67ca2b9e659-kube-api-access-rlt44" (OuterVolumeSpecName: "kube-api-access-rlt44") pod "fec845ed-6569-41a3-8a07-f67ca2b9e659" (UID: "fec845ed-6569-41a3-8a07-f67ca2b9e659"). InnerVolumeSpecName "kube-api-access-rlt44". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:54:17.059631 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.059580    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlt44\" (UniqueName: \"kubernetes.io/projected/fec845ed-6569-41a3-8a07-f67ca2b9e659-kube-api-access-rlt44\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:54:17.059631 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.059627    2568 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/fec845ed-6569-41a3-8a07-f67ca2b9e659-extensions-socket-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:54:17.246014 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.245924    2568 generic.go:356] "Generic (PLEG): container finished" podID="fec845ed-6569-41a3-8a07-f67ca2b9e659" containerID="11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182" exitCode=0
Jul 28 19:54:17.246014 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.245973    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"
Jul 28 19:54:17.246014 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.246004    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs" event={"ID":"fec845ed-6569-41a3-8a07-f67ca2b9e659","Type":"ContainerDied","Data":"11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182"}
Jul 28 19:54:17.246226 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.246058    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs" event={"ID":"fec845ed-6569-41a3-8a07-f67ca2b9e659","Type":"ContainerDied","Data":"5f223f0f9810b1d22e4cd3e731d2a452be2eb5d0a52a28c5a81e770b4574a270"}
Jul 28 19:54:17.246226 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.246079    2568 scope.go:117] "RemoveContainer" containerID="11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182"
Jul 28 19:54:17.255674 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.255648    2568 scope.go:117] "RemoveContainer" containerID="11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182"
Jul 28 19:54:17.255913 ip-10-0-128-255 kubenswrapper[2568]: E0728 19:54:17.255893    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182\": container with ID starting with 11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182 not found: ID does not exist" containerID="11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182"
Jul 28 19:54:17.255973 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.255929    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182"} err="failed to get container status \"11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182\": rpc error: code = NotFound desc = could not find container \"11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182\": container with ID starting with 11c7fc2f1d5b7ea319bce8840fc2e413d0b556fe00348e934a0f10382ad26182 not found: ID does not exist"
Jul 28 19:54:17.268919 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.268897    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"]
Jul 28 19:54:17.273412 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.273390    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-njfjs"]
Jul 28 19:54:17.675765 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:17.675730    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fec845ed-6569-41a3-8a07-f67ca2b9e659" path="/var/lib/kubelet/pods/fec845ed-6569-41a3-8a07-f67ca2b9e659/volumes"
Jul 28 19:54:18.803740 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:18.803705    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:29.443467 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:29.443426    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:54:47.607064 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:47.607032    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:54:47.611048 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:54:47.611025    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:58:17.063042 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.063007    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"]
Jul 28 19:58:17.063551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.063506    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fec845ed-6569-41a3-8a07-f67ca2b9e659" containerName="manager"
Jul 28 19:58:17.063551 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.063525    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fec845ed-6569-41a3-8a07-f67ca2b9e659" containerName="manager"
Jul 28 19:58:17.063635 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.063600    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fec845ed-6569-41a3-8a07-f67ca2b9e659" containerName="manager"
Jul 28 19:58:17.066507 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.066489    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.069242 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.069217    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-5d56be1d-kserve-self-signed-certs\""
Jul 28 19:58:17.086271 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.086247    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"]
Jul 28 19:58:17.138452 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.138415    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:17.146097 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.146075    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-home\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.146193 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.146110    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-dshm\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.146193 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.146128    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmdct\" (UniqueName: \"kubernetes.io/projected/d130f4ed-fefb-4555-8439-53e0970f780e-kube-api-access-zmdct\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.146193 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.146150    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d130f4ed-fefb-4555-8439-53e0970f780e-tls-certs\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.146316 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.146224    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-model-cache\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.146316 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.146248    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-tmp-dir\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247243 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247205    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-home\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247439 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247266    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-dshm\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247439 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247291    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmdct\" (UniqueName: \"kubernetes.io/projected/d130f4ed-fefb-4555-8439-53e0970f780e-kube-api-access-zmdct\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247439 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247347    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d130f4ed-fefb-4555-8439-53e0970f780e-tls-certs\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247439 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247395    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-model-cache\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247439 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247419    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-tmp-dir\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247726 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247630    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-home\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247845 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247800    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-model-cache\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.247964 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.247845    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-tmp-dir\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.249647 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.249629    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-dshm\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.249901 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.249882    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d130f4ed-fefb-4555-8439-53e0970f780e-tls-certs\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.255513 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.255491    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmdct\" (UniqueName: \"kubernetes.io/projected/d130f4ed-fefb-4555-8439-53e0970f780e-kube-api-access-zmdct\") pod \"e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") " pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.375716 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.375626    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:17.506008 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:17.505980    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"]
Jul 28 19:58:17.507805 ip-10-0-128-255 kubenswrapper[2568]: W0728 19:58:17.507777    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd130f4ed_fefb_4555_8439_53e0970f780e.slice/crio-f6b29e3da33a58327c2408117e443ad5e320f5a716f0c2fc4c1123b83afcbe91 WatchSource:0}: Error finding container f6b29e3da33a58327c2408117e443ad5e320f5a716f0c2fc4c1123b83afcbe91: Status 404 returned error can't find the container with id f6b29e3da33a58327c2408117e443ad5e320f5a716f0c2fc4c1123b83afcbe91
Jul 28 19:58:18.360849 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:18.360814    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk" event={"ID":"d130f4ed-fefb-4555-8439-53e0970f780e","Type":"ContainerStarted","Data":"17a143d4d38502aa948f19f31e036af9fe7944a3bbd438d64344bd6744d2e1c9"}
Jul 28 19:58:18.360849 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:18.360857    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk" event={"ID":"d130f4ed-fefb-4555-8439-53e0970f780e","Type":"ContainerStarted","Data":"f6b29e3da33a58327c2408117e443ad5e320f5a716f0c2fc4c1123b83afcbe91"}
Jul 28 19:58:18.377799 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:18.377749    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk" podStartSLOduration=1.377732704 podStartE2EDuration="1.377732704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 19:58:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 19:58:18.376797384 +0000 UTC m=+1411.188359246" watchObservedRunningTime="2026-07-28 19:58:18.377732704 +0000 UTC m=+1411.189294567"
Jul 28 19:58:27.376449 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:27.376357    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:27.376449 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:27.376415    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:27.388637 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:27.388606    2568 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:27.420103 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:27.420069    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:58:27.780193 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:27.780161    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:32.321694 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:32.321657    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:51.334931 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:51.334857    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"]
Jul 28 19:58:51.355832 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:51.355805    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:58:52.092605 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:58:52.092571    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 19:59:06.354018 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.353977    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk" podUID="d130f4ed-fefb-4555-8439-53e0970f780e" containerName="main" containerID="cri-o://17a143d4d38502aa948f19f31e036af9fe7944a3bbd438d64344bd6744d2e1c9" gracePeriod=45
Jul 28 19:59:06.588862 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.588829    2568 generic.go:356] "Generic (PLEG): container finished" podID="d130f4ed-fefb-4555-8439-53e0970f780e" containerID="17a143d4d38502aa948f19f31e036af9fe7944a3bbd438d64344bd6744d2e1c9" exitCode=0
Jul 28 19:59:06.589028 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.588905    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk" event={"ID":"d130f4ed-fefb-4555-8439-53e0970f780e","Type":"ContainerDied","Data":"17a143d4d38502aa948f19f31e036af9fe7944a3bbd438d64344bd6744d2e1c9"}
Jul 28 19:59:06.607070 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.607006    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:59:06.708309 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708271    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d130f4ed-fefb-4555-8439-53e0970f780e-tls-certs\") pod \"d130f4ed-fefb-4555-8439-53e0970f780e\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") "
Jul 28 19:59:06.708488 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708317    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-dshm\") pod \"d130f4ed-fefb-4555-8439-53e0970f780e\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") "
Jul 28 19:59:06.708488 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708407    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-tmp-dir\") pod \"d130f4ed-fefb-4555-8439-53e0970f780e\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") "
Jul 28 19:59:06.708488 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708465    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmdct\" (UniqueName: \"kubernetes.io/projected/d130f4ed-fefb-4555-8439-53e0970f780e-kube-api-access-zmdct\") pod \"d130f4ed-fefb-4555-8439-53e0970f780e\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") "
Jul 28 19:59:06.708632 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708516    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-home\") pod \"d130f4ed-fefb-4555-8439-53e0970f780e\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") "
Jul 28 19:59:06.708632 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708544    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-model-cache\") pod \"d130f4ed-fefb-4555-8439-53e0970f780e\" (UID: \"d130f4ed-fefb-4555-8439-53e0970f780e\") "
Jul 28 19:59:06.708735 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708660    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d130f4ed-fefb-4555-8439-53e0970f780e" (UID: "d130f4ed-fefb-4555-8439-53e0970f780e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:59:06.708793 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708761    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-home" (OuterVolumeSpecName: "home") pod "d130f4ed-fefb-4555-8439-53e0970f780e" (UID: "d130f4ed-fefb-4555-8439-53e0970f780e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:59:06.708872 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708848    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-model-cache" (OuterVolumeSpecName: "model-cache") pod "d130f4ed-fefb-4555-8439-53e0970f780e" (UID: "d130f4ed-fefb-4555-8439-53e0970f780e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:59:06.708967 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708948    2568 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-tmp-dir\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:59:06.709017 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708973    2568 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-home\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:59:06.709017 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.708986    2568 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-model-cache\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:59:06.710599 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.710564    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d130f4ed-fefb-4555-8439-53e0970f780e-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d130f4ed-fefb-4555-8439-53e0970f780e" (UID: "d130f4ed-fefb-4555-8439-53e0970f780e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 19:59:06.710700 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.710655    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-dshm" (OuterVolumeSpecName: "dshm") pod "d130f4ed-fefb-4555-8439-53e0970f780e" (UID: "d130f4ed-fefb-4555-8439-53e0970f780e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 28 19:59:06.710700 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.710672    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d130f4ed-fefb-4555-8439-53e0970f780e-kube-api-access-zmdct" (OuterVolumeSpecName: "kube-api-access-zmdct") pod "d130f4ed-fefb-4555-8439-53e0970f780e" (UID: "d130f4ed-fefb-4555-8439-53e0970f780e"). InnerVolumeSpecName "kube-api-access-zmdct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 19:59:06.809966 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.809933    2568 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d130f4ed-fefb-4555-8439-53e0970f780e-tls-certs\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:59:06.809966 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.809960    2568 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d130f4ed-fefb-4555-8439-53e0970f780e-dshm\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:59:06.809966 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:06.809970    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmdct\" (UniqueName: \"kubernetes.io/projected/d130f4ed-fefb-4555-8439-53e0970f780e-kube-api-access-zmdct\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 19:59:07.595072 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:07.595034    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk" event={"ID":"d130f4ed-fefb-4555-8439-53e0970f780e","Type":"ContainerDied","Data":"f6b29e3da33a58327c2408117e443ad5e320f5a716f0c2fc4c1123b83afcbe91"}
Jul 28 19:59:07.595072 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:07.595067    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"
Jul 28 19:59:07.595658 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:07.595091    2568 scope.go:117] "RemoveContainer" containerID="17a143d4d38502aa948f19f31e036af9fe7944a3bbd438d64344bd6744d2e1c9"
Jul 28 19:59:07.616395 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:07.616370    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"]
Jul 28 19:59:07.619826 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:07.619798    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-5d56be1d-kserve-589b7f98d7-rjsmk"]
Jul 28 19:59:07.682304 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:07.682269    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d130f4ed-fefb-4555-8439-53e0970f780e" path="/var/lib/kubelet/pods/d130f4ed-fefb-4555-8439-53e0970f780e/volumes"
Jul 28 19:59:47.637860 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:47.637832    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 19:59:47.645380 ip-10-0-128-255 kubenswrapper[2568]: I0728 19:59:47.645349    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:00:00.151158 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.151111    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42"]
Jul 28 20:00:00.151799 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.151650    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d130f4ed-fefb-4555-8439-53e0970f780e" containerName="main"
Jul 28 20:00:00.151799 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.151672    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d130f4ed-fefb-4555-8439-53e0970f780e" containerName="main"
Jul 28 20:00:00.151799 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.151770    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d130f4ed-fefb-4555-8439-53e0970f780e" containerName="main"
Jul 28 20:00:00.155041 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.155019    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42"
Jul 28 20:00:00.157573 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.157551    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:00:00.161839 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.161813    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42"]
Jul 28 20:00:00.166130 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.166107    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42"
Jul 28 20:00:00.510480 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.510384    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42"]
Jul 28 20:00:00.513473 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:00:00.513442    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb99094c3_2863_45bb_8319_ddfdf1c2e5dd.slice/crio-7dfe8826c040fff35ba5cc0cbd86cd1cb88da2636432ca4009db56d25dd077e6 WatchSource:0}: Error finding container 7dfe8826c040fff35ba5cc0cbd86cd1cb88da2636432ca4009db56d25dd077e6: Status 404 returned error can't find the container with id 7dfe8826c040fff35ba5cc0cbd86cd1cb88da2636432ca4009db56d25dd077e6
Jul 28 20:00:00.515573 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.515552    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:00:00.848740 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:00.848659    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42" event={"ID":"b99094c3-2863-45bb-8319-ddfdf1c2e5dd","Type":"ContainerStarted","Data":"7dfe8826c040fff35ba5cc0cbd86cd1cb88da2636432ca4009db56d25dd077e6"}
Jul 28 20:00:03.864981 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:03.864892    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42" event={"ID":"b99094c3-2863-45bb-8319-ddfdf1c2e5dd","Type":"ContainerStarted","Data":"fa34dffae6bb7c10211fcd9c7d9929a85524c8672815c8ef6a27d1b98f383984"}
Jul 28 20:00:03.880942 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:03.880893    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42" podStartSLOduration=0.988368701 podStartE2EDuration="3.880878877s" totalImagesPullingTime="2.892510176s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:00:03.878853052 +0000 UTC m=+1516.690414915" watchObservedRunningTime="2026-07-28 20:00:03.880878877 +0000 UTC m=+1516.692440739"
Jul 28 20:00:04.871612 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:04.871577    2568 generic.go:356] "Generic (PLEG): container finished" podID="b99094c3-2863-45bb-8319-ddfdf1c2e5dd" containerID="fa34dffae6bb7c10211fcd9c7d9929a85524c8672815c8ef6a27d1b98f383984" exitCode=0
Jul 28 20:00:04.871999 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:04.871646    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42" event={"ID":"b99094c3-2863-45bb-8319-ddfdf1c2e5dd","Type":"ContainerDied","Data":"fa34dffae6bb7c10211fcd9c7d9929a85524c8672815c8ef6a27d1b98f383984"}
Jul 28 20:00:06.009623 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:06.009595    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42"
Jul 28 20:00:06.882383 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:06.882352    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42"
Jul 28 20:00:06.882383 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:06.882372    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754480-hjh42" event={"ID":"b99094c3-2863-45bb-8319-ddfdf1c2e5dd","Type":"ContainerDied","Data":"7dfe8826c040fff35ba5cc0cbd86cd1cb88da2636432ca4009db56d25dd077e6"}
Jul 28 20:00:06.882605 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:00:06.882420    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7dfe8826c040fff35ba5cc0cbd86cd1cb88da2636432ca4009db56d25dd077e6"
Jul 28 20:01:05.905878 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:01:05.905841    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:12.225457 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:12.225375    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:36.294086 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:36.294048    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:40.151853 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:40.151822    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:40.252176 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:40.252142    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:45.845430 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:45.845396    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:45.919047 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:45.919003    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:46.089624 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:46.089591    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:46.430239 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:46.430210    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:46.893026 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:46.892993    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:48.919518 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:48.919487    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:49.027866 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.027830    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:49.154407 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.154376    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:49.425206 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.425172    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:49.695049 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.694966    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:49.800313 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.800267    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:02:49.825114 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.825066    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"]
Jul 28 20:02:49.825473 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.825457    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b99094c3-2863-45bb-8319-ddfdf1c2e5dd" containerName="cleanup"
Jul 28 20:02:49.825473 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.825472    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b99094c3-2863-45bb-8319-ddfdf1c2e5dd" containerName="cleanup"
Jul 28 20:02:49.825648 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.825538    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b99094c3-2863-45bb-8319-ddfdf1c2e5dd" containerName="cleanup"
Jul 28 20:02:49.828542 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.828521    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:49.831114 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.831091    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-fc68dda7\""
Jul 28 20:02:49.831209 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.831114    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-fc68dda7-dockercfg-5p7rz\""
Jul 28 20:02:49.841675 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.841650    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"]
Jul 28 20:02:49.889061 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.889025    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2zvq\" (UniqueName: \"kubernetes.io/projected/42cc8343-b119-4dfa-8dfd-38f2a6f29338-kube-api-access-t2zvq\") pod \"payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586\" (UID: \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\") " pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:49.889184 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.889162    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42cc8343-b119-4dfa-8dfd-38f2a6f29338-plugins-config-volume\") pod \"payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586\" (UID: \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\") " pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:49.990417 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.990316    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t2zvq\" (UniqueName: \"kubernetes.io/projected/42cc8343-b119-4dfa-8dfd-38f2a6f29338-kube-api-access-t2zvq\") pod \"payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586\" (UID: \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\") " pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:49.990820 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.990464    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42cc8343-b119-4dfa-8dfd-38f2a6f29338-plugins-config-volume\") pod \"payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586\" (UID: \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\") " pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:49.991177 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.991157    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42cc8343-b119-4dfa-8dfd-38f2a6f29338-plugins-config-volume\") pod \"payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586\" (UID: \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\") " pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:49.998155 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:49.998126    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2zvq\" (UniqueName: \"kubernetes.io/projected/42cc8343-b119-4dfa-8dfd-38f2a6f29338-kube-api-access-t2zvq\") pod \"payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586\" (UID: \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\") " pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:50.140293 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:50.140241    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:50.278900 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:50.278874    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"]
Jul 28 20:02:50.280964 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:02:50.280939    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod42cc8343_b119_4dfa_8dfd_38f2a6f29338.slice/crio-b5d3aa84f792998e6d2f45db0dc830c7d8eb20304294b7c3c502beac75c9456e WatchSource:0}: Error finding container b5d3aa84f792998e6d2f45db0dc830c7d8eb20304294b7c3c502beac75c9456e: Status 404 returned error can't find the container with id b5d3aa84f792998e6d2f45db0dc830c7d8eb20304294b7c3c502beac75c9456e
Jul 28 20:02:50.651385 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:50.651352    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586" event={"ID":"42cc8343-b119-4dfa-8dfd-38f2a6f29338","Type":"ContainerStarted","Data":"b5d3aa84f792998e6d2f45db0dc830c7d8eb20304294b7c3c502beac75c9456e"}
Jul 28 20:02:52.662797 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:52.662767    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586" event={"ID":"42cc8343-b119-4dfa-8dfd-38f2a6f29338","Type":"ContainerStarted","Data":"30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b"}
Jul 28 20:02:52.663191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:52.662830    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:02:52.680216 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:02:52.680167    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586" podStartSLOduration=1.416646165 podStartE2EDuration="3.680151762s" totalImagesPullingTime="2.263505597s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:02:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:02:52.678845958 +0000 UTC m=+1685.490407842" watchObservedRunningTime="2026-07-28 20:02:52.680151762 +0000 UTC m=+1685.491713626"
Jul 28 20:03:03.669732 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:03.669699    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:03:14.491535 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.491498    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"]
Jul 28 20:03:14.500637 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.500601    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"
Jul 28 20:03:14.501414 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.501381    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"]
Jul 28 20:03:14.503245 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.503224    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:03:14.503361 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.503225    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:03:14.607372 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.607312    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac491b17-d64f-4a18-b4a8-41df4b3b71e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj\" (UID: \"ac491b17-d64f-4a18-b4a8-41df4b3b71e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"
Jul 28 20:03:14.708546 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.708512    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac491b17-d64f-4a18-b4a8-41df4b3b71e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj\" (UID: \"ac491b17-d64f-4a18-b4a8-41df4b3b71e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"
Jul 28 20:03:14.709107 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.709086    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac491b17-d64f-4a18-b4a8-41df4b3b71e5-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj\" (UID: \"ac491b17-d64f-4a18-b4a8-41df4b3b71e5\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"
Jul 28 20:03:14.811841 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:14.811768    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"
Jul 28 20:03:15.155957 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:15.155929    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"]
Jul 28 20:03:15.158891 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:03:15.158863    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac491b17_d64f_4a18_b4a8_41df4b3b71e5.slice/crio-34d7c883c65554375f76b9d6788cf20a1ede6a607c9304583bcbcc0673fdc7a4 WatchSource:0}: Error finding container 34d7c883c65554375f76b9d6788cf20a1ede6a607c9304583bcbcc0673fdc7a4: Status 404 returned error can't find the container with id 34d7c883c65554375f76b9d6788cf20a1ede6a607c9304583bcbcc0673fdc7a4
Jul 28 20:03:15.773900 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:15.773796    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj" event={"ID":"ac491b17-d64f-4a18-b4a8-41df4b3b71e5","Type":"ContainerStarted","Data":"3d6390a8c6fb0b58b32738aa9fd601667afa3bf497dbd38a6265ff5c08532bed"}
Jul 28 20:03:15.773900 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:15.773857    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj" event={"ID":"ac491b17-d64f-4a18-b4a8-41df4b3b71e5","Type":"ContainerStarted","Data":"34d7c883c65554375f76b9d6788cf20a1ede6a607c9304583bcbcc0673fdc7a4"}
Jul 28 20:03:15.789751 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:15.789692    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj" podStartSLOduration=1.78967376 podStartE2EDuration="1.78967376s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:03:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:03:15.787527142 +0000 UTC m=+1708.599089024" watchObservedRunningTime="2026-07-28 20:03:15.78967376 +0000 UTC m=+1708.601235624"
Jul 28 20:03:16.779666 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:16.779627    2568 generic.go:356] "Generic (PLEG): container finished" podID="ac491b17-d64f-4a18-b4a8-41df4b3b71e5" containerID="3d6390a8c6fb0b58b32738aa9fd601667afa3bf497dbd38a6265ff5c08532bed" exitCode=0
Jul 28 20:03:16.780096 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:16.779686    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj" event={"ID":"ac491b17-d64f-4a18-b4a8-41df4b3b71e5","Type":"ContainerDied","Data":"3d6390a8c6fb0b58b32738aa9fd601667afa3bf497dbd38a6265ff5c08532bed"}
Jul 28 20:03:17.917049 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:17.917021    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"
Jul 28 20:03:18.038880 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:18.038788    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac491b17-d64f-4a18-b4a8-41df4b3b71e5-maas-api-service-ca\") pod \"ac491b17-d64f-4a18-b4a8-41df4b3b71e5\" (UID: \"ac491b17-d64f-4a18-b4a8-41df4b3b71e5\") "
Jul 28 20:03:18.039192 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:18.039162    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ac491b17-d64f-4a18-b4a8-41df4b3b71e5-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ac491b17-d64f-4a18-b4a8-41df4b3b71e5" (UID: "ac491b17-d64f-4a18-b4a8-41df4b3b71e5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:18.140158 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:18.140118    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ac491b17-d64f-4a18-b4a8-41df4b3b71e5-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:18.790560 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:18.790524    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"
Jul 28 20:03:18.790708 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:18.790576    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj" event={"ID":"ac491b17-d64f-4a18-b4a8-41df4b3b71e5","Type":"ContainerDied","Data":"34d7c883c65554375f76b9d6788cf20a1ede6a607c9304583bcbcc0673fdc7a4"}
Jul 28 20:03:18.790708 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:18.790623    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="34d7c883c65554375f76b9d6788cf20a1ede6a607c9304583bcbcc0673fdc7a4"
Jul 28 20:03:19.803085 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:19.803053    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:19.830005 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:19.829974    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"]
Jul 28 20:03:19.830216 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:19.830189    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586" podUID="42cc8343-b119-4dfa-8dfd-38f2a6f29338" containerName="payload-processing" containerID="cri-o://30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b" gracePeriod=30
Jul 28 20:03:19.901655 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:19.901614    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:19.973575 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:19.973541    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"]
Jul 28 20:03:19.976219 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:19.976187    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-fc68dda7-b7jmj"]
Jul 28 20:03:19.977147 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:19.977005    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:20.037300 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.037269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:20.089734 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.089704    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:03:20.151845 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.151810    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:20.260541 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.260507    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2zvq\" (UniqueName: \"kubernetes.io/projected/42cc8343-b119-4dfa-8dfd-38f2a6f29338-kube-api-access-t2zvq\") pod \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\" (UID: \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\") "
Jul 28 20:03:20.260735 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.260561    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42cc8343-b119-4dfa-8dfd-38f2a6f29338-plugins-config-volume\") pod \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\" (UID: \"42cc8343-b119-4dfa-8dfd-38f2a6f29338\") "
Jul 28 20:03:20.260967 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.260942    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/42cc8343-b119-4dfa-8dfd-38f2a6f29338-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "42cc8343-b119-4dfa-8dfd-38f2a6f29338" (UID: "42cc8343-b119-4dfa-8dfd-38f2a6f29338"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:20.262876 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.262844    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/42cc8343-b119-4dfa-8dfd-38f2a6f29338-kube-api-access-t2zvq" (OuterVolumeSpecName: "kube-api-access-t2zvq") pod "42cc8343-b119-4dfa-8dfd-38f2a6f29338" (UID: "42cc8343-b119-4dfa-8dfd-38f2a6f29338"). InnerVolumeSpecName "kube-api-access-t2zvq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:03:20.361507 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.361430    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2zvq\" (UniqueName: \"kubernetes.io/projected/42cc8343-b119-4dfa-8dfd-38f2a6f29338-kube-api-access-t2zvq\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:20.361507 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.361459    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/42cc8343-b119-4dfa-8dfd-38f2a6f29338-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:20.394726 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.394688    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:20.642369 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.642316    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:20.801836 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.801805    2568 generic.go:356] "Generic (PLEG): container finished" podID="42cc8343-b119-4dfa-8dfd-38f2a6f29338" containerID="30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b" exitCode=0
Jul 28 20:03:20.801999 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.801851    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"
Jul 28 20:03:20.801999 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.801901    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586" event={"ID":"42cc8343-b119-4dfa-8dfd-38f2a6f29338","Type":"ContainerDied","Data":"30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b"}
Jul 28 20:03:20.801999 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.801953    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586" event={"ID":"42cc8343-b119-4dfa-8dfd-38f2a6f29338","Type":"ContainerDied","Data":"b5d3aa84f792998e6d2f45db0dc830c7d8eb20304294b7c3c502beac75c9456e"}
Jul 28 20:03:20.801999 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.801982    2568 scope.go:117] "RemoveContainer" containerID="30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b"
Jul 28 20:03:20.812282 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.812265    2568 scope.go:117] "RemoveContainer" containerID="30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b"
Jul 28 20:03:20.812550 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:03:20.812519    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b\": container with ID starting with 30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b not found: ID does not exist" containerID="30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b"
Jul 28 20:03:20.812591 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.812543    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b"} err="failed to get container status \"30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b\": rpc error: code = NotFound desc = could not find container \"30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b\": container with ID starting with 30cb94cecc4212a48a7bf879a9b6828a3b57bd1b9dc741168eab7638d608732b not found: ID does not exist"
Jul 28 20:03:20.825259 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.825237    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"]
Jul 28 20:03:20.828967 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:20.828946    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-fc68dda7-7494ff76d6-x8586"]
Jul 28 20:03:21.676139 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:21.676105    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="42cc8343-b119-4dfa-8dfd-38f2a6f29338" path="/var/lib/kubelet/pods/42cc8343-b119-4dfa-8dfd-38f2a6f29338/volumes"
Jul 28 20:03:21.676458 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:21.676444    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ac491b17-d64f-4a18-b4a8-41df4b3b71e5" path="/var/lib/kubelet/pods/ac491b17-d64f-4a18-b4a8-41df4b3b71e5/volumes"
Jul 28 20:03:21.981140 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:21.981029    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:22.082252 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:22.082217    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:23.222037 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.222003    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:23.305685 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.305647    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"]
Jul 28 20:03:23.306000 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.305988    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ac491b17-d64f-4a18-b4a8-41df4b3b71e5" containerName="revoke-keys"
Jul 28 20:03:23.306044 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.306001    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ac491b17-d64f-4a18-b4a8-41df4b3b71e5" containerName="revoke-keys"
Jul 28 20:03:23.306044 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.306013    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="42cc8343-b119-4dfa-8dfd-38f2a6f29338" containerName="payload-processing"
Jul 28 20:03:23.306044 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.306019    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="42cc8343-b119-4dfa-8dfd-38f2a6f29338" containerName="payload-processing"
Jul 28 20:03:23.306139 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.306086    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ac491b17-d64f-4a18-b4a8-41df4b3b71e5" containerName="revoke-keys"
Jul 28 20:03:23.306139 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.306097    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="42cc8343-b119-4dfa-8dfd-38f2a6f29338" containerName="payload-processing"
Jul 28 20:03:23.310465 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.310444    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.312995 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.312969    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:03:23.313191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.312973    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:03:23.313806 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.313787    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:03:23.313924 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.313795    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-e98c78a1-serving-cert\""
Jul 28 20:03:23.317167 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.317141    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"]
Jul 28 20:03:23.340579 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.340540    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:23.489499 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.489392    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wx8l6\" (UniqueName: \"kubernetes.io/projected/2f585427-913b-4644-a886-9e47c228d75d-kube-api-access-wx8l6\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.489499 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.489484    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f585427-913b-4644-a886-9e47c228d75d-maas-api-tls\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.489732 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.489563    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f585427-913b-4644-a886-9e47c228d75d-openshift-service-ca\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.516537 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.516503    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:23.590596 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.590562    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wx8l6\" (UniqueName: \"kubernetes.io/projected/2f585427-913b-4644-a886-9e47c228d75d-kube-api-access-wx8l6\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.590765 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.590612    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f585427-913b-4644-a886-9e47c228d75d-maas-api-tls\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.590765 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.590649    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f585427-913b-4644-a886-9e47c228d75d-openshift-service-ca\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.591227 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.591207    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f585427-913b-4644-a886-9e47c228d75d-openshift-service-ca\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.593040 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.593017    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f585427-913b-4644-a886-9e47c228d75d-maas-api-tls\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.598175 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.598149    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wx8l6\" (UniqueName: \"kubernetes.io/projected/2f585427-913b-4644-a886-9e47c228d75d-kube-api-access-wx8l6\") pod \"maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.623050 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.623017    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:23.775604 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.775522    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"]
Jul 28 20:03:23.778451 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:03:23.778423    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f585427_913b_4644_a886_9e47c228d75d.slice/crio-91ba38ed801366752650c2d7f500cd8ff4e7c114d5e6b46dd69652464999b985 WatchSource:0}: Error finding container 91ba38ed801366752650c2d7f500cd8ff4e7c114d5e6b46dd69652464999b985: Status 404 returned error can't find the container with id 91ba38ed801366752650c2d7f500cd8ff4e7c114d5e6b46dd69652464999b985
Jul 28 20:03:23.818766 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.818732    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5" event={"ID":"2f585427-913b-4644-a886-9e47c228d75d","Type":"ContainerStarted","Data":"91ba38ed801366752650c2d7f500cd8ff4e7c114d5e6b46dd69652464999b985"}
Jul 28 20:03:23.839469 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.839438    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:23.934502 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.934453    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:23.935333 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.935297    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"]
Jul 28 20:03:23.947581 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.947545    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:23.949188 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.949159    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"]
Jul 28 20:03:23.950106 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.950085    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-e98c78a1\""
Jul 28 20:03:23.982425 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.982396    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"]
Jul 28 20:03:23.985800 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.985784    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:23.988165 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.988145    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-e98c78a1-dockercfg-l9h6z\""
Jul 28 20:03:23.994299 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.994277    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkjsw\" (UniqueName: \"kubernetes.io/projected/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-kube-api-access-qkjsw\") pod \"payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx\" (UID: \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:23.994487 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.994341    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-np684\" (UniqueName: \"kubernetes.io/projected/315b9050-02de-4fbb-9f58-d55e851feb0f-kube-api-access-np684\") pod \"payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj\" (UID: \"315b9050-02de-4fbb-9f58-d55e851feb0f\") " pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:23.994487 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.994399    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx\" (UID: \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:23.994613 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.994494    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/315b9050-02de-4fbb-9f58-d55e851feb0f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj\" (UID: \"315b9050-02de-4fbb-9f58-d55e851feb0f\") " pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:23.998120 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:23.998096    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"]
Jul 28 20:03:24.095699 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.095625    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkjsw\" (UniqueName: \"kubernetes.io/projected/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-kube-api-access-qkjsw\") pod \"payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx\" (UID: \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:24.095699 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.095659    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-np684\" (UniqueName: \"kubernetes.io/projected/315b9050-02de-4fbb-9f58-d55e851feb0f-kube-api-access-np684\") pod \"payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj\" (UID: \"315b9050-02de-4fbb-9f58-d55e851feb0f\") " pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:24.095699 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.095682    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx\" (UID: \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:24.095924 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.095713    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/315b9050-02de-4fbb-9f58-d55e851feb0f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj\" (UID: \"315b9050-02de-4fbb-9f58-d55e851feb0f\") " pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:24.096405 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.096387    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/315b9050-02de-4fbb-9f58-d55e851feb0f-plugins-config-volume\") pod \"payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj\" (UID: \"315b9050-02de-4fbb-9f58-d55e851feb0f\") " pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:24.096457 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.096399    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx\" (UID: \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:24.103317 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.103288    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkjsw\" (UniqueName: \"kubernetes.io/projected/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-kube-api-access-qkjsw\") pod \"payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx\" (UID: \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:24.103440 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.103348    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-np684\" (UniqueName: \"kubernetes.io/projected/315b9050-02de-4fbb-9f58-d55e851feb0f-kube-api-access-np684\") pod \"payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj\" (UID: \"315b9050-02de-4fbb-9f58-d55e851feb0f\") " pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:24.261634 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.261599    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:24.298544 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.298512    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:24.431829 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.431789    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"]
Jul 28 20:03:24.436392 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:03:24.436357    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddacb559d_eebf_4f46_bf8d_1cbcaf67d56d.slice/crio-ccb7debcd7bb52ffff723ce3b06ba660672ceb945a7ab201c3d5cd815c485777 WatchSource:0}: Error finding container ccb7debcd7bb52ffff723ce3b06ba660672ceb945a7ab201c3d5cd815c485777: Status 404 returned error can't find the container with id ccb7debcd7bb52ffff723ce3b06ba660672ceb945a7ab201c3d5cd815c485777
Jul 28 20:03:24.465349 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.465285    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"]
Jul 28 20:03:24.475660 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:03:24.475629    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod315b9050_02de_4fbb_9f58_d55e851feb0f.slice/crio-24fd0b1dd9447036ecfb7994f0012ec6868c2f36f6e67a59ec8be714e79c1b5c WatchSource:0}: Error finding container 24fd0b1dd9447036ecfb7994f0012ec6868c2f36f6e67a59ec8be714e79c1b5c: Status 404 returned error can't find the container with id 24fd0b1dd9447036ecfb7994f0012ec6868c2f36f6e67a59ec8be714e79c1b5c
Jul 28 20:03:24.826201 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.826129    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj" event={"ID":"315b9050-02de-4fbb-9f58-d55e851feb0f","Type":"ContainerStarted","Data":"4c326ef42d0cf5da8d0f230d13daf9041a1ecb3222d215e7fe5e93b39ec409cd"}
Jul 28 20:03:24.826201 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.826197    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:24.826499 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.826221    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj" event={"ID":"315b9050-02de-4fbb-9f58-d55e851feb0f","Type":"ContainerStarted","Data":"24fd0b1dd9447036ecfb7994f0012ec6868c2f36f6e67a59ec8be714e79c1b5c"}
Jul 28 20:03:24.828209 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.828125    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx" event={"ID":"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d","Type":"ContainerStarted","Data":"cda6f420613a0eb6cc0666a0a563bba42d63e510c723a5439493ffddedcc71f2"}
Jul 28 20:03:24.828209 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.828168    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx" event={"ID":"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d","Type":"ContainerStarted","Data":"ccb7debcd7bb52ffff723ce3b06ba660672ceb945a7ab201c3d5cd815c485777"}
Jul 28 20:03:24.828421 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.828216    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:24.844754 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.844712    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj" podStartSLOduration=1.8446965450000001 podStartE2EDuration="1.844696545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:03:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:03:24.842339355 +0000 UTC m=+1717.653901247" watchObservedRunningTime="2026-07-28 20:03:24.844696545 +0000 UTC m=+1717.656258407"
Jul 28 20:03:24.857898 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:24.857849    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx" podStartSLOduration=1.857829369 podStartE2EDuration="1.857829369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:03:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:03:24.855462501 +0000 UTC m=+1717.667024366" watchObservedRunningTime="2026-07-28 20:03:24.857829369 +0000 UTC m=+1717.669391233"
Jul 28 20:03:26.840734 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:26.840636    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5" event={"ID":"2f585427-913b-4644-a886-9e47c228d75d","Type":"ContainerStarted","Data":"c136aafcaa4a5c235b387ea3d9decfc9d292c1dafaae1199eefd670302af8617"}
Jul 28 20:03:26.840734 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:26.840716    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:26.859663 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:26.859613    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5" podStartSLOduration=1.213084957 podStartE2EDuration="3.859598342s" totalImagesPullingTime="2.646513385s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:03:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:03:26.858013605 +0000 UTC m=+1719.669575469" watchObservedRunningTime="2026-07-28 20:03:26.859598342 +0000 UTC m=+1719.671160205"
Jul 28 20:03:32.850791 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:32.850754    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:35.836983 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:35.836943    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:35.837436 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:35.837012    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:48.265562 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.265524    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"]
Jul 28 20:03:48.270123 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.270105    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"
Jul 28 20:03:48.272878 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.272858    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:03:48.276972 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.276946    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"]
Jul 28 20:03:48.407915 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.407875    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26549db7-a4bb-41cf-8e02-09a9cf1890c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd\" (UID: \"26549db7-a4bb-41cf-8e02-09a9cf1890c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"
Jul 28 20:03:48.508458 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.508421    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26549db7-a4bb-41cf-8e02-09a9cf1890c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd\" (UID: \"26549db7-a4bb-41cf-8e02-09a9cf1890c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"
Jul 28 20:03:48.509107 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.509085    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26549db7-a4bb-41cf-8e02-09a9cf1890c2-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd\" (UID: \"26549db7-a4bb-41cf-8e02-09a9cf1890c2\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"
Jul 28 20:03:48.582587 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.582504    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"
Jul 28 20:03:48.710360 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.710315    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"]
Jul 28 20:03:48.712889 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:03:48.712862    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26549db7_a4bb_41cf_8e02_09a9cf1890c2.slice/crio-6682d1c0e6cbe601700efdccf3697f4243ac9ea8cffa57135ff5299fd62409e6 WatchSource:0}: Error finding container 6682d1c0e6cbe601700efdccf3697f4243ac9ea8cffa57135ff5299fd62409e6: Status 404 returned error can't find the container with id 6682d1c0e6cbe601700efdccf3697f4243ac9ea8cffa57135ff5299fd62409e6
Jul 28 20:03:48.955572 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:48.955537    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd" event={"ID":"26549db7-a4bb-41cf-8e02-09a9cf1890c2","Type":"ContainerStarted","Data":"6682d1c0e6cbe601700efdccf3697f4243ac9ea8cffa57135ff5299fd62409e6"}
Jul 28 20:03:49.962194 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:49.962161    2568 generic.go:356] "Generic (PLEG): container finished" podID="26549db7-a4bb-41cf-8e02-09a9cf1890c2" containerID="a8814df34add1f9e6db083839832be59a10f703b87f414f7e55ea0285fe8d1ce" exitCode=0
Jul 28 20:03:49.962591 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:49.962247    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd" event={"ID":"26549db7-a4bb-41cf-8e02-09a9cf1890c2","Type":"ContainerDied","Data":"a8814df34add1f9e6db083839832be59a10f703b87f414f7e55ea0285fe8d1ce"}
Jul 28 20:03:51.101828 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:51.101801    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"
Jul 28 20:03:51.234641 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:51.234555    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26549db7-a4bb-41cf-8e02-09a9cf1890c2-maas-api-service-ca\") pod \"26549db7-a4bb-41cf-8e02-09a9cf1890c2\" (UID: \"26549db7-a4bb-41cf-8e02-09a9cf1890c2\") "
Jul 28 20:03:51.234913 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:51.234891    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26549db7-a4bb-41cf-8e02-09a9cf1890c2-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "26549db7-a4bb-41cf-8e02-09a9cf1890c2" (UID: "26549db7-a4bb-41cf-8e02-09a9cf1890c2"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:51.335559 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:51.335518    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/26549db7-a4bb-41cf-8e02-09a9cf1890c2-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:51.973397 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:51.973302    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"
Jul 28 20:03:51.973397 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:51.973360    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd" event={"ID":"26549db7-a4bb-41cf-8e02-09a9cf1890c2","Type":"ContainerDied","Data":"6682d1c0e6cbe601700efdccf3697f4243ac9ea8cffa57135ff5299fd62409e6"}
Jul 28 20:03:51.973571 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:51.973406    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6682d1c0e6cbe601700efdccf3697f4243ac9ea8cffa57135ff5299fd62409e6"
Jul 28 20:03:54.770402 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.770365    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"]
Jul 28 20:03:54.770908 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.770557    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5" podUID="2f585427-913b-4644-a886-9e47c228d75d" containerName="maas-api" containerID="cri-o://c136aafcaa4a5c235b387ea3d9decfc9d292c1dafaae1199eefd670302af8617" gracePeriod=30
Jul 28 20:03:54.797548 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.797524    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:54.826165 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.826132    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"]
Jul 28 20:03:54.826451 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.826400    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj" podUID="315b9050-02de-4fbb-9f58-d55e851feb0f" containerName="payload-processing" containerID="cri-o://4c326ef42d0cf5da8d0f230d13daf9041a1ecb3222d215e7fe5e93b39ec409cd" gracePeriod=30
Jul 28 20:03:54.836612 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.836583    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"]
Jul 28 20:03:54.836867 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.836839    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx" podUID="dacb559d-eebf-4f46-bf8d-1cbcaf67d56d" containerName="payload-pre-processing" containerID="cri-o://cda6f420613a0eb6cc0666a0a563bba42d63e510c723a5439493ffddedcc71f2" gracePeriod=30
Jul 28 20:03:54.869045 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.869012    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:54.937884 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.937855    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:54.993803 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.991213    2568 generic.go:356] "Generic (PLEG): container finished" podID="2f585427-913b-4644-a886-9e47c228d75d" containerID="c136aafcaa4a5c235b387ea3d9decfc9d292c1dafaae1199eefd670302af8617" exitCode=0
Jul 28 20:03:54.993803 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.991417    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5" event={"ID":"2f585427-913b-4644-a886-9e47c228d75d","Type":"ContainerDied","Data":"c136aafcaa4a5c235b387ea3d9decfc9d292c1dafaae1199eefd670302af8617"}
Jul 28 20:03:54.994546 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.994452    2568 generic.go:356] "Generic (PLEG): container finished" podID="315b9050-02de-4fbb-9f58-d55e851feb0f" containerID="4c326ef42d0cf5da8d0f230d13daf9041a1ecb3222d215e7fe5e93b39ec409cd" exitCode=0
Jul 28 20:03:54.994546 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.994529    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj" event={"ID":"315b9050-02de-4fbb-9f58-d55e851feb0f","Type":"ContainerDied","Data":"4c326ef42d0cf5da8d0f230d13daf9041a1ecb3222d215e7fe5e93b39ec409cd"}
Jul 28 20:03:54.995723 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.995672    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"]
Jul 28 20:03:54.997035 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.996907    2568 generic.go:356] "Generic (PLEG): container finished" podID="dacb559d-eebf-4f46-bf8d-1cbcaf67d56d" containerID="cda6f420613a0eb6cc0666a0a563bba42d63e510c723a5439493ffddedcc71f2" exitCode=0
Jul 28 20:03:54.997035 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.996973    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx" event={"ID":"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d","Type":"ContainerDied","Data":"cda6f420613a0eb6cc0666a0a563bba42d63e510c723a5439493ffddedcc71f2"}
Jul 28 20:03:54.997626 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:54.997606    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-e98c78a1-rkdrd"]
Jul 28 20:03:55.003507 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.003450    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:55.115848 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.115816    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:55.137097 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.137053    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:55.150330 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.150295    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:55.159367 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.159294    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:55.272418 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.272384    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-plugins-config-volume\") pod \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\" (UID: \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\") "
Jul 28 20:03:55.272629 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.272454    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f585427-913b-4644-a886-9e47c228d75d-maas-api-tls\") pod \"2f585427-913b-4644-a886-9e47c228d75d\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") "
Jul 28 20:03:55.272629 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.272492    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wx8l6\" (UniqueName: \"kubernetes.io/projected/2f585427-913b-4644-a886-9e47c228d75d-kube-api-access-wx8l6\") pod \"2f585427-913b-4644-a886-9e47c228d75d\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") "
Jul 28 20:03:55.272629 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.272513    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkjsw\" (UniqueName: \"kubernetes.io/projected/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-kube-api-access-qkjsw\") pod \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\" (UID: \"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d\") "
Jul 28 20:03:55.272796 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.272656    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f585427-913b-4644-a886-9e47c228d75d-openshift-service-ca\") pod \"2f585427-913b-4644-a886-9e47c228d75d\" (UID: \"2f585427-913b-4644-a886-9e47c228d75d\") "
Jul 28 20:03:55.272796 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.272725    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/315b9050-02de-4fbb-9f58-d55e851feb0f-plugins-config-volume\") pod \"315b9050-02de-4fbb-9f58-d55e851feb0f\" (UID: \"315b9050-02de-4fbb-9f58-d55e851feb0f\") "
Jul 28 20:03:55.272796 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.272761    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-np684\" (UniqueName: \"kubernetes.io/projected/315b9050-02de-4fbb-9f58-d55e851feb0f-kube-api-access-np684\") pod \"315b9050-02de-4fbb-9f58-d55e851feb0f\" (UID: \"315b9050-02de-4fbb-9f58-d55e851feb0f\") "
Jul 28 20:03:55.272961 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.272801    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "dacb559d-eebf-4f46-bf8d-1cbcaf67d56d" (UID: "dacb559d-eebf-4f46-bf8d-1cbcaf67d56d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:55.273057 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.273026    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f585427-913b-4644-a886-9e47c228d75d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2f585427-913b-4644-a886-9e47c228d75d" (UID: "2f585427-913b-4644-a886-9e47c228d75d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:55.273146 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.273123    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/315b9050-02de-4fbb-9f58-d55e851feb0f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "315b9050-02de-4fbb-9f58-d55e851feb0f" (UID: "315b9050-02de-4fbb-9f58-d55e851feb0f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:03:55.273225 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.273141    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2f585427-913b-4644-a886-9e47c228d75d-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:55.273225 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.273164    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:55.274788 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.274765    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-kube-api-access-qkjsw" (OuterVolumeSpecName: "kube-api-access-qkjsw") pod "dacb559d-eebf-4f46-bf8d-1cbcaf67d56d" (UID: "dacb559d-eebf-4f46-bf8d-1cbcaf67d56d"). InnerVolumeSpecName "kube-api-access-qkjsw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:03:55.274875 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.274793    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/315b9050-02de-4fbb-9f58-d55e851feb0f-kube-api-access-np684" (OuterVolumeSpecName: "kube-api-access-np684") pod "315b9050-02de-4fbb-9f58-d55e851feb0f" (UID: "315b9050-02de-4fbb-9f58-d55e851feb0f"). InnerVolumeSpecName "kube-api-access-np684". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:03:55.274875 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.274812    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f585427-913b-4644-a886-9e47c228d75d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2f585427-913b-4644-a886-9e47c228d75d" (UID: "2f585427-913b-4644-a886-9e47c228d75d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:03:55.274875 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.274854    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f585427-913b-4644-a886-9e47c228d75d-kube-api-access-wx8l6" (OuterVolumeSpecName: "kube-api-access-wx8l6") pod "2f585427-913b-4644-a886-9e47c228d75d" (UID: "2f585427-913b-4644-a886-9e47c228d75d"). InnerVolumeSpecName "kube-api-access-wx8l6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:03:55.374120 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.374043    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/315b9050-02de-4fbb-9f58-d55e851feb0f-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:55.374120 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.374071    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-np684\" (UniqueName: \"kubernetes.io/projected/315b9050-02de-4fbb-9f58-d55e851feb0f-kube-api-access-np684\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:55.374120 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.374082    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2f585427-913b-4644-a886-9e47c228d75d-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:55.374120 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.374091    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wx8l6\" (UniqueName: \"kubernetes.io/projected/2f585427-913b-4644-a886-9e47c228d75d-kube-api-access-wx8l6\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:55.374120 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.374099    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkjsw\" (UniqueName: \"kubernetes.io/projected/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d-kube-api-access-qkjsw\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:03:55.677206 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:55.677171    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26549db7-a4bb-41cf-8e02-09a9cf1890c2" path="/var/lib/kubelet/pods/26549db7-a4bb-41cf-8e02-09a9cf1890c2/volumes"
Jul 28 20:03:56.003991 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.003920    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"
Jul 28 20:03:56.003991 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.003950    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5" event={"ID":"2f585427-913b-4644-a886-9e47c228d75d","Type":"ContainerDied","Data":"91ba38ed801366752650c2d7f500cd8ff4e7c114d5e6b46dd69652464999b985"}
Jul 28 20:03:56.004505 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.004011    2568 scope.go:117] "RemoveContainer" containerID="c136aafcaa4a5c235b387ea3d9decfc9d292c1dafaae1199eefd670302af8617"
Jul 28 20:03:56.005810 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.005785    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj" event={"ID":"315b9050-02de-4fbb-9f58-d55e851feb0f","Type":"ContainerDied","Data":"24fd0b1dd9447036ecfb7994f0012ec6868c2f36f6e67a59ec8be714e79c1b5c"}
Jul 28 20:03:56.006017 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.005998    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"
Jul 28 20:03:56.007513 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.007484    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx" event={"ID":"dacb559d-eebf-4f46-bf8d-1cbcaf67d56d","Type":"ContainerDied","Data":"ccb7debcd7bb52ffff723ce3b06ba660672ceb945a7ab201c3d5cd815c485777"}
Jul 28 20:03:56.007636 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.007549    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"
Jul 28 20:03:56.014726 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.014705    2568 scope.go:117] "RemoveContainer" containerID="4c326ef42d0cf5da8d0f230d13daf9041a1ecb3222d215e7fe5e93b39ec409cd"
Jul 28 20:03:56.022939 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.022915    2568 scope.go:117] "RemoveContainer" containerID="cda6f420613a0eb6cc0666a0a563bba42d63e510c723a5439493ffddedcc71f2"
Jul 28 20:03:56.028187 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.027961    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"]
Jul 28 20:03:56.031012 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.030986    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-e98c78a1-6f565b98dd-r25lx"]
Jul 28 20:03:56.042205 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.042182    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"]
Jul 28 20:03:56.045841 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.045819    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-e98c78a1-75fdbc967c-4jhbj"]
Jul 28 20:03:56.062338 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.058405    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"]
Jul 28 20:03:56.065130 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.065104    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-e98c78a1-674c4c66dd-6czk5"]
Jul 28 20:03:56.297225 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.297139    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:56.365710 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.365675    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:56.599796 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:56.599708    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:57.676736 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:57.676701    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f585427-913b-4644-a886-9e47c228d75d" path="/var/lib/kubelet/pods/2f585427-913b-4644-a886-9e47c228d75d/volumes"
Jul 28 20:03:57.677224 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:57.677207    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="315b9050-02de-4fbb-9f58-d55e851feb0f" path="/var/lib/kubelet/pods/315b9050-02de-4fbb-9f58-d55e851feb0f/volumes"
Jul 28 20:03:57.677617 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:57.677596    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dacb559d-eebf-4f46-bf8d-1cbcaf67d56d" path="/var/lib/kubelet/pods/dacb559d-eebf-4f46-bf8d-1cbcaf67d56d/volumes"
Jul 28 20:03:58.291503 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.291459    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:58.373426 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.373390    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:58.374590 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.374570    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"]
Jul 28 20:03:58.374970 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.374932    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="315b9050-02de-4fbb-9f58-d55e851feb0f" containerName="payload-processing"
Jul 28 20:03:58.375012 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.374972    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="315b9050-02de-4fbb-9f58-d55e851feb0f" containerName="payload-processing"
Jul 28 20:03:58.375012 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.374983    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dacb559d-eebf-4f46-bf8d-1cbcaf67d56d" containerName="payload-pre-processing"
Jul 28 20:03:58.375012 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.374990    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dacb559d-eebf-4f46-bf8d-1cbcaf67d56d" containerName="payload-pre-processing"
Jul 28 20:03:58.375012 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.374998    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f585427-913b-4644-a886-9e47c228d75d" containerName="maas-api"
Jul 28 20:03:58.375012 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.375004    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f585427-913b-4644-a886-9e47c228d75d" containerName="maas-api"
Jul 28 20:03:58.375012 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.375013    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26549db7-a4bb-41cf-8e02-09a9cf1890c2" containerName="revoke-keys"
Jul 28 20:03:58.375191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.375018    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="26549db7-a4bb-41cf-8e02-09a9cf1890c2" containerName="revoke-keys"
Jul 28 20:03:58.375191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.375079    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="315b9050-02de-4fbb-9f58-d55e851feb0f" containerName="payload-processing"
Jul 28 20:03:58.375191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.375088    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="26549db7-a4bb-41cf-8e02-09a9cf1890c2" containerName="revoke-keys"
Jul 28 20:03:58.375191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.375096    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dacb559d-eebf-4f46-bf8d-1cbcaf67d56d" containerName="payload-pre-processing"
Jul 28 20:03:58.375191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.375104    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f585427-913b-4644-a886-9e47c228d75d" containerName="maas-api"
Jul 28 20:03:58.384396 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.384371    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.387265 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.387242    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:03:58.388858 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.388836    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:03:58.389018 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.388996    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-8b9a5873-serving-cert\""
Jul 28 20:03:58.389797 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.389773    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:03:58.391515 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.391489    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"]
Jul 28 20:03:58.441291 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.441262    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:58.499645 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.499610    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.499797 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.499757    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lllw2\" (UniqueName: \"kubernetes.io/projected/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-kube-api-access-lllw2\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.499797 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.499786    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-openshift-service-ca\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.601171 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.601080    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lllw2\" (UniqueName: \"kubernetes.io/projected/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-kube-api-access-lllw2\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.601171 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.601131    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-openshift-service-ca\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.601171 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.601167    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.601499 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:03:58.601305    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-serving-cert: secret "maas-api-e2e-ait-8b9a5873-serving-cert" not found
Jul 28 20:03:58.601499 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:03:58.601402    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls podName:f709db3a-6ac4-4be5-b94d-f2ff059e23b7 nodeName:}" failed. No retries permitted until 2026-07-28 20:03:59.101380314 +0000 UTC m=+1751.912942172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls") pod "maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w" (UID: "f709db3a-6ac4-4be5-b94d-f2ff059e23b7") : secret "maas-api-e2e-ait-8b9a5873-serving-cert" not found
Jul 28 20:03:58.601857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.601835    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-openshift-service-ca\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.610495 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.610460    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lllw2\" (UniqueName: \"kubernetes.io/projected/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-kube-api-access-lllw2\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:58.835637 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:58.835586    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:59.083827 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.083797    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:59.105888 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.105848    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:59.108390 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.108313    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls\") pod \"maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:59.183921 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.183884    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:03:59.190530 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.190500    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"]
Jul 28 20:03:59.194654 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.194634    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:03:59.200985 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.200961    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-8b9a5873\""
Jul 28 20:03:59.214937 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.214915    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"]
Jul 28 20:03:59.236217 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.236194    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"]
Jul 28 20:03:59.239959 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.239942    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:03:59.242660 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.242632    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-8b9a5873-dockercfg-57zdf\""
Jul 28 20:03:59.248599 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.248576    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"]
Jul 28 20:03:59.299634 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.299607    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:03:59.307135 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.307112    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb\" (UID: \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:03:59.307218 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.307165    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvrjc\" (UniqueName: \"kubernetes.io/projected/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-kube-api-access-wvrjc\") pod \"payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb\" (UID: \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:03:59.407920 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.407882    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb\" (UID: \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:03:59.408162 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.407947    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvrjc\" (UniqueName: \"kubernetes.io/projected/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-kube-api-access-wvrjc\") pod \"payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb\" (UID: \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:03:59.408162 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.407994    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1d1a3-09c0-458d-a160-0065c8266ccc-plugins-config-volume\") pod \"payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7\" (UID: \"40c1d1a3-09c0-458d-a160-0065c8266ccc\") " pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:03:59.408162 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.408013    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d2px\" (UniqueName: \"kubernetes.io/projected/40c1d1a3-09c0-458d-a160-0065c8266ccc-kube-api-access-4d2px\") pod \"payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7\" (UID: \"40c1d1a3-09c0-458d-a160-0065c8266ccc\") " pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:03:59.408750 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.408728    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb\" (UID: \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:03:59.417224 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.417196    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvrjc\" (UniqueName: \"kubernetes.io/projected/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-kube-api-access-wvrjc\") pod \"payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb\" (UID: \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:03:59.433168 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.433138    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"]
Jul 28 20:03:59.435588 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:03:59.435556    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf709db3a_6ac4_4be5_b94d_f2ff059e23b7.slice/crio-e235c8f453eac73f714215152c75639f9c593b46467d2d50872f0a87e0662534 WatchSource:0}: Error finding container e235c8f453eac73f714215152c75639f9c593b46467d2d50872f0a87e0662534: Status 404 returned error can't find the container with id e235c8f453eac73f714215152c75639f9c593b46467d2d50872f0a87e0662534
Jul 28 20:03:59.504884 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.504848    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:03:59.509211 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.509184    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1d1a3-09c0-458d-a160-0065c8266ccc-plugins-config-volume\") pod \"payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7\" (UID: \"40c1d1a3-09c0-458d-a160-0065c8266ccc\") " pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:03:59.509304 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.509220    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d2px\" (UniqueName: \"kubernetes.io/projected/40c1d1a3-09c0-458d-a160-0065c8266ccc-kube-api-access-4d2px\") pod \"payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7\" (UID: \"40c1d1a3-09c0-458d-a160-0065c8266ccc\") " pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:03:59.509821 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.509800    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1d1a3-09c0-458d-a160-0065c8266ccc-plugins-config-volume\") pod \"payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7\" (UID: \"40c1d1a3-09c0-458d-a160-0065c8266ccc\") " pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:03:59.518409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.518382    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d2px\" (UniqueName: \"kubernetes.io/projected/40c1d1a3-09c0-458d-a160-0065c8266ccc-kube-api-access-4d2px\") pod \"payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7\" (UID: \"40c1d1a3-09c0-458d-a160-0065c8266ccc\") " pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:03:59.551461 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.551421    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:03:59.652844 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.652816    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"]
Jul 28 20:03:59.654432 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:03:59.654400    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0f79eb0_6692_4d9e_9152_d7bd8d922f47.slice/crio-46852887175a85a67b7340b922b40c4ea46f9a7fc9b1bf5087a0f0759fc2eda5 WatchSource:0}: Error finding container 46852887175a85a67b7340b922b40c4ea46f9a7fc9b1bf5087a0f0759fc2eda5: Status 404 returned error can't find the container with id 46852887175a85a67b7340b922b40c4ea46f9a7fc9b1bf5087a0f0759fc2eda5
Jul 28 20:03:59.715701 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:03:59.715675    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"]
Jul 28 20:03:59.717773 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:03:59.717737    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40c1d1a3_09c0_458d_a160_0065c8266ccc.slice/crio-56a5bebe3bc8e9a1bbd85a0a62ddb44b3168ea7e732900df57bed03078602dd4 WatchSource:0}: Error finding container 56a5bebe3bc8e9a1bbd85a0a62ddb44b3168ea7e732900df57bed03078602dd4: Status 404 returned error can't find the container with id 56a5bebe3bc8e9a1bbd85a0a62ddb44b3168ea7e732900df57bed03078602dd4
Jul 28 20:04:00.031743 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.031694    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w" event={"ID":"f709db3a-6ac4-4be5-b94d-f2ff059e23b7","Type":"ContainerStarted","Data":"e235c8f453eac73f714215152c75639f9c593b46467d2d50872f0a87e0662534"}
Jul 28 20:04:00.033238 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.033210    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb" event={"ID":"b0f79eb0-6692-4d9e-9152-d7bd8d922f47","Type":"ContainerStarted","Data":"0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c"}
Jul 28 20:04:00.033384 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.033261    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb" event={"ID":"b0f79eb0-6692-4d9e-9152-d7bd8d922f47","Type":"ContainerStarted","Data":"46852887175a85a67b7340b922b40c4ea46f9a7fc9b1bf5087a0f0759fc2eda5"}
Jul 28 20:04:00.033384 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.033290    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:04:00.034716 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.034690    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7" event={"ID":"40c1d1a3-09c0-458d-a160-0065c8266ccc","Type":"ContainerStarted","Data":"78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042"}
Jul 28 20:04:00.034807 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.034726    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7" event={"ID":"40c1d1a3-09c0-458d-a160-0065c8266ccc","Type":"ContainerStarted","Data":"56a5bebe3bc8e9a1bbd85a0a62ddb44b3168ea7e732900df57bed03078602dd4"}
Jul 28 20:04:00.034807 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.034766    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:04:00.052007 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.051959    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb" podStartSLOduration=1.05194431 podStartE2EDuration="1.05194431s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:04:00.051182327 +0000 UTC m=+1752.862744203" watchObservedRunningTime="2026-07-28 20:04:00.05194431 +0000 UTC m=+1752.863506173"
Jul 28 20:04:00.070288 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:00.070247    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7" podStartSLOduration=1.070232935 podStartE2EDuration="1.070232935s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:03:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:04:00.069450067 +0000 UTC m=+1752.881011929" watchObservedRunningTime="2026-07-28 20:04:00.070232935 +0000 UTC m=+1752.881794776"
Jul 28 20:04:01.041769 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:01.041676    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w" event={"ID":"f709db3a-6ac4-4be5-b94d-f2ff059e23b7","Type":"ContainerStarted","Data":"4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563"}
Jul 28 20:04:01.042222 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:01.041864    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:04:01.060341 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:01.060285    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w" podStartSLOduration=1.8203886790000001 podStartE2EDuration="3.060272435s" totalImagesPullingTime="1.239883756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:03:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:04:01.059044887 +0000 UTC m=+1753.870606750" watchObservedRunningTime="2026-07-28 20:04:01.060272435 +0000 UTC m=+1753.871834330"
Jul 28 20:04:07.051783 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:07.051751    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:04:11.044082 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:11.044049    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:04:11.044481 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:11.044308    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:04:24.144436 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.144401    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"]
Jul 28 20:04:24.147925 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.147908    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"
Jul 28 20:04:24.150446 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.150427    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:04:24.154969 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.154943    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"]
Jul 28 20:04:24.228929 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.228893    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66df0829-cddb-4d4e-958f-7f6a4a1dbead-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd\" (UID: \"66df0829-cddb-4d4e-958f-7f6a4a1dbead\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"
Jul 28 20:04:24.330273 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.330236    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66df0829-cddb-4d4e-958f-7f6a4a1dbead-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd\" (UID: \"66df0829-cddb-4d4e-958f-7f6a4a1dbead\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"
Jul 28 20:04:24.330999 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.330978    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66df0829-cddb-4d4e-958f-7f6a4a1dbead-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd\" (UID: \"66df0829-cddb-4d4e-958f-7f6a4a1dbead\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"
Jul 28 20:04:24.460852 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.460769    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"
Jul 28 20:04:24.590871 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:24.590844    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"]
Jul 28 20:04:24.593089 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:04:24.593062    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66df0829_cddb_4d4e_958f_7f6a4a1dbead.slice/crio-c18d83a421c7e0d61e4ec8f6dd46839a03ad5b71e19ce9bb7bb7b99a190e94e2 WatchSource:0}: Error finding container c18d83a421c7e0d61e4ec8f6dd46839a03ad5b71e19ce9bb7bb7b99a190e94e2: Status 404 returned error can't find the container with id c18d83a421c7e0d61e4ec8f6dd46839a03ad5b71e19ce9bb7bb7b99a190e94e2
Jul 28 20:04:25.162224 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:25.162183    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd" event={"ID":"66df0829-cddb-4d4e-958f-7f6a4a1dbead","Type":"ContainerStarted","Data":"cd5b7f2564c303e23d7214822ee8680d80adf49c3c8fd6c9f41d3fd9439a1896"}
Jul 28 20:04:25.162640 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:25.162239    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd" event={"ID":"66df0829-cddb-4d4e-958f-7f6a4a1dbead","Type":"ContainerStarted","Data":"c18d83a421c7e0d61e4ec8f6dd46839a03ad5b71e19ce9bb7bb7b99a190e94e2"}
Jul 28 20:04:25.177136 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:25.177083    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd" podStartSLOduration=1.177066322 podStartE2EDuration="1.177066322s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:04:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:04:25.176388794 +0000 UTC m=+1777.987950678" watchObservedRunningTime="2026-07-28 20:04:25.177066322 +0000 UTC m=+1777.988628186"
Jul 28 20:04:26.168811 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:26.168775    2568 generic.go:356] "Generic (PLEG): container finished" podID="66df0829-cddb-4d4e-958f-7f6a4a1dbead" containerID="cd5b7f2564c303e23d7214822ee8680d80adf49c3c8fd6c9f41d3fd9439a1896" exitCode=0
Jul 28 20:04:26.169277 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:26.168824    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd" event={"ID":"66df0829-cddb-4d4e-958f-7f6a4a1dbead","Type":"ContainerDied","Data":"cd5b7f2564c303e23d7214822ee8680d80adf49c3c8fd6c9f41d3fd9439a1896"}
Jul 28 20:04:27.311899 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:27.311872    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"
Jul 28 20:04:27.357526 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:27.357494    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66df0829-cddb-4d4e-958f-7f6a4a1dbead-maas-api-service-ca\") pod \"66df0829-cddb-4d4e-958f-7f6a4a1dbead\" (UID: \"66df0829-cddb-4d4e-958f-7f6a4a1dbead\") "
Jul 28 20:04:27.357842 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:27.357821    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/66df0829-cddb-4d4e-958f-7f6a4a1dbead-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "66df0829-cddb-4d4e-958f-7f6a4a1dbead" (UID: "66df0829-cddb-4d4e-958f-7f6a4a1dbead"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:27.357928 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:27.357916    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/66df0829-cddb-4d4e-958f-7f6a4a1dbead-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:28.179848 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:28.179819    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"
Jul 28 20:04:28.180013 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:28.179862    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd" event={"ID":"66df0829-cddb-4d4e-958f-7f6a4a1dbead","Type":"ContainerDied","Data":"c18d83a421c7e0d61e4ec8f6dd46839a03ad5b71e19ce9bb7bb7b99a190e94e2"}
Jul 28 20:04:28.180013 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:28.179912    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c18d83a421c7e0d61e4ec8f6dd46839a03ad5b71e19ce9bb7bb7b99a190e94e2"
Jul 28 20:04:29.093070 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.093029    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"]
Jul 28 20:04:29.093571 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.093284    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w" podUID="f709db3a-6ac4-4be5-b94d-f2ff059e23b7" containerName="maas-api" containerID="cri-o://4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563" gracePeriod=30
Jul 28 20:04:29.120990 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.120964    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:29.142870 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.142837    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"]
Jul 28 20:04:29.143147 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.143091    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7" podUID="40c1d1a3-09c0-458d-a160-0065c8266ccc" containerName="payload-processing" containerID="cri-o://78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042" gracePeriod=30
Jul 28 20:04:29.154381 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.154352    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"]
Jul 28 20:04:29.154612 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.154589    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb" podUID="b0f79eb0-6692-4d9e-9152-d7bd8d922f47" containerName="payload-pre-processing" containerID="cri-o://0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c" gracePeriod=30
Jul 28 20:04:29.185615 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.185584    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:29.249166 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.249123    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:29.301936 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.301881    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"]
Jul 28 20:04:29.304390 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.304361    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-8b9a5873-sggvd"]
Jul 28 20:04:29.317157 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.317122    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:29.423072 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.423048    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:04:29.428655 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.428635    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:04:29.438856 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.438836    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:04:29.480562 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.480531    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls\") pod \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") "
Jul 28 20:04:29.480562 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.480567    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-plugins-config-volume\") pod \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\" (UID: \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\") "
Jul 28 20:04:29.480847 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.480599    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1d1a3-09c0-458d-a160-0065c8266ccc-plugins-config-volume\") pod \"40c1d1a3-09c0-458d-a160-0065c8266ccc\" (UID: \"40c1d1a3-09c0-458d-a160-0065c8266ccc\") "
Jul 28 20:04:29.480847 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.480623    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-openshift-service-ca\") pod \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") "
Jul 28 20:04:29.480847 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.480669    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvrjc\" (UniqueName: \"kubernetes.io/projected/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-kube-api-access-wvrjc\") pod \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\" (UID: \"b0f79eb0-6692-4d9e-9152-d7bd8d922f47\") "
Jul 28 20:04:29.480847 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.480744    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d2px\" (UniqueName: \"kubernetes.io/projected/40c1d1a3-09c0-458d-a160-0065c8266ccc-kube-api-access-4d2px\") pod \"40c1d1a3-09c0-458d-a160-0065c8266ccc\" (UID: \"40c1d1a3-09c0-458d-a160-0065c8266ccc\") "
Jul 28 20:04:29.480847 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.480773    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lllw2\" (UniqueName: \"kubernetes.io/projected/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-kube-api-access-lllw2\") pod \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\" (UID: \"f709db3a-6ac4-4be5-b94d-f2ff059e23b7\") "
Jul 28 20:04:29.481127 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.480988    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40c1d1a3-09c0-458d-a160-0065c8266ccc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "40c1d1a3-09c0-458d-a160-0065c8266ccc" (UID: "40c1d1a3-09c0-458d-a160-0065c8266ccc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:29.481127 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.481025    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f709db3a-6ac4-4be5-b94d-f2ff059e23b7" (UID: "f709db3a-6ac4-4be5-b94d-f2ff059e23b7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:29.481127 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.481097    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40c1d1a3-09c0-458d-a160-0065c8266ccc-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:29.481127 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.481115    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:29.481433 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.481393    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b0f79eb0-6692-4d9e-9152-d7bd8d922f47" (UID: "b0f79eb0-6692-4d9e-9152-d7bd8d922f47"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:29.482988 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.482961    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f709db3a-6ac4-4be5-b94d-f2ff059e23b7" (UID: "f709db3a-6ac4-4be5-b94d-f2ff059e23b7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:04:29.482988 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.482973    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-kube-api-access-lllw2" (OuterVolumeSpecName: "kube-api-access-lllw2") pod "f709db3a-6ac4-4be5-b94d-f2ff059e23b7" (UID: "f709db3a-6ac4-4be5-b94d-f2ff059e23b7"). InnerVolumeSpecName "kube-api-access-lllw2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:04:29.483288 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.483265    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-kube-api-access-wvrjc" (OuterVolumeSpecName: "kube-api-access-wvrjc") pod "b0f79eb0-6692-4d9e-9152-d7bd8d922f47" (UID: "b0f79eb0-6692-4d9e-9152-d7bd8d922f47"). InnerVolumeSpecName "kube-api-access-wvrjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:04:29.483288 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.483274    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40c1d1a3-09c0-458d-a160-0065c8266ccc-kube-api-access-4d2px" (OuterVolumeSpecName: "kube-api-access-4d2px") pod "40c1d1a3-09c0-458d-a160-0065c8266ccc" (UID: "40c1d1a3-09c0-458d-a160-0065c8266ccc"). InnerVolumeSpecName "kube-api-access-4d2px". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:04:29.582017 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.581978    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d2px\" (UniqueName: \"kubernetes.io/projected/40c1d1a3-09c0-458d-a160-0065c8266ccc-kube-api-access-4d2px\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:29.582017 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.582010    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lllw2\" (UniqueName: \"kubernetes.io/projected/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-kube-api-access-lllw2\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:29.582017 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.582020    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f709db3a-6ac4-4be5-b94d-f2ff059e23b7-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:29.582247 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.582029    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:29.582247 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.582038    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvrjc\" (UniqueName: \"kubernetes.io/projected/b0f79eb0-6692-4d9e-9152-d7bd8d922f47-kube-api-access-wvrjc\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:29.675633 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:29.675597    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="66df0829-cddb-4d4e-958f-7f6a4a1dbead" path="/var/lib/kubelet/pods/66df0829-cddb-4d4e-958f-7f6a4a1dbead/volumes"
Jul 28 20:04:30.191681 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.191643    2568 generic.go:356] "Generic (PLEG): container finished" podID="40c1d1a3-09c0-458d-a160-0065c8266ccc" containerID="78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042" exitCode=0
Jul 28 20:04:30.192235 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.191690    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"
Jul 28 20:04:30.192235 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.191738    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7" event={"ID":"40c1d1a3-09c0-458d-a160-0065c8266ccc","Type":"ContainerDied","Data":"78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042"}
Jul 28 20:04:30.192235 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.191803    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7" event={"ID":"40c1d1a3-09c0-458d-a160-0065c8266ccc","Type":"ContainerDied","Data":"56a5bebe3bc8e9a1bbd85a0a62ddb44b3168ea7e732900df57bed03078602dd4"}
Jul 28 20:04:30.192235 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.191837    2568 scope.go:117] "RemoveContainer" containerID="78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042"
Jul 28 20:04:30.193177 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.193155    2568 generic.go:356] "Generic (PLEG): container finished" podID="f709db3a-6ac4-4be5-b94d-f2ff059e23b7" containerID="4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563" exitCode=0
Jul 28 20:04:30.193256 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.193211    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"
Jul 28 20:04:30.193425 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.193279    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w" event={"ID":"f709db3a-6ac4-4be5-b94d-f2ff059e23b7","Type":"ContainerDied","Data":"4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563"}
Jul 28 20:04:30.193425 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.193349    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w" event={"ID":"f709db3a-6ac4-4be5-b94d-f2ff059e23b7","Type":"ContainerDied","Data":"e235c8f453eac73f714215152c75639f9c593b46467d2d50872f0a87e0662534"}
Jul 28 20:04:30.194776 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.194752    2568 generic.go:356] "Generic (PLEG): container finished" podID="b0f79eb0-6692-4d9e-9152-d7bd8d922f47" containerID="0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c" exitCode=0
Jul 28 20:04:30.194888 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.194794    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"
Jul 28 20:04:30.194888 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.194827    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb" event={"ID":"b0f79eb0-6692-4d9e-9152-d7bd8d922f47","Type":"ContainerDied","Data":"0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c"}
Jul 28 20:04:30.194888 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.194863    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb" event={"ID":"b0f79eb0-6692-4d9e-9152-d7bd8d922f47","Type":"ContainerDied","Data":"46852887175a85a67b7340b922b40c4ea46f9a7fc9b1bf5087a0f0759fc2eda5"}
Jul 28 20:04:30.201501 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.201484    2568 scope.go:117] "RemoveContainer" containerID="78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042"
Jul 28 20:04:30.201728 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:04:30.201708    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042\": container with ID starting with 78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042 not found: ID does not exist" containerID="78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042"
Jul 28 20:04:30.201778 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.201735    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042"} err="failed to get container status \"78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042\": rpc error: code = NotFound desc = could not find container \"78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042\": container with ID starting with 78b02bac82c6edb3620c193c9df6b1155356f9ee9bcca91c5780c0c93af7c042 not found: ID does not exist"
Jul 28 20:04:30.201778 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.201753    2568 scope.go:117] "RemoveContainer" containerID="4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563"
Jul 28 20:04:30.209356 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.209339    2568 scope.go:117] "RemoveContainer" containerID="4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563"
Jul 28 20:04:30.209593 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:04:30.209575    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563\": container with ID starting with 4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563 not found: ID does not exist" containerID="4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563"
Jul 28 20:04:30.209647 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.209598    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563"} err="failed to get container status \"4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563\": rpc error: code = NotFound desc = could not find container \"4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563\": container with ID starting with 4ea6565d7afa9f63a5baa68917d09b5100f562ac63d70568a952602fd440f563 not found: ID does not exist"
Jul 28 20:04:30.209647 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.209612    2568 scope.go:117] "RemoveContainer" containerID="0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c"
Jul 28 20:04:30.214467 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.214423    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"]
Jul 28 20:04:30.216454 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.216432    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-8b9a5873-f5fdbc4d4-cw49w"]
Jul 28 20:04:30.218056 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.218042    2568 scope.go:117] "RemoveContainer" containerID="0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c"
Jul 28 20:04:30.218296 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:04:30.218279    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c\": container with ID starting with 0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c not found: ID does not exist" containerID="0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c"
Jul 28 20:04:30.218352 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.218310    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c"} err="failed to get container status \"0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c\": rpc error: code = NotFound desc = could not find container \"0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c\": container with ID starting with 0cab7017eeddfd02ab0ef3964b715122ca2af8b2d01d4cd2ae077ea3341ae18c not found: ID does not exist"
Jul 28 20:04:30.227678 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.227653    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"]
Jul 28 20:04:30.231714 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.231687    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-8b9a5873-6c79b7f846-9rbh7"]
Jul 28 20:04:30.242540 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.242515    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"]
Jul 28 20:04:30.246007 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.245987    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-8b9a5873-85f87d688d-9nnlb"]
Jul 28 20:04:30.546575 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.546540    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:30.761788 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:30.761747    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:31.073581 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:31.073543    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:31.379580 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:31.379494    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:31.513608 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:31.513566    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:31.677256 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:31.677218    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40c1d1a3-09c0-458d-a160-0065c8266ccc" path="/var/lib/kubelet/pods/40c1d1a3-09c0-458d-a160-0065c8266ccc/volumes"
Jul 28 20:04:31.677619 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:31.677604    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b0f79eb0-6692-4d9e-9152-d7bd8d922f47" path="/var/lib/kubelet/pods/b0f79eb0-6692-4d9e-9152-d7bd8d922f47/volumes"
Jul 28 20:04:31.677947 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:31.677935    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f709db3a-6ac4-4be5-b94d-f2ff059e23b7" path="/var/lib/kubelet/pods/f709db3a-6ac4-4be5-b94d-f2ff059e23b7/volumes"
Jul 28 20:04:33.505878 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:33.505838    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:33.622802 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:33.622768    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:34.521495 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521459    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"]
Jul 28 20:04:34.521857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521794    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40c1d1a3-09c0-458d-a160-0065c8266ccc" containerName="payload-processing"
Jul 28 20:04:34.521857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521804    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="40c1d1a3-09c0-458d-a160-0065c8266ccc" containerName="payload-processing"
Jul 28 20:04:34.521857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521817    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b0f79eb0-6692-4d9e-9152-d7bd8d922f47" containerName="payload-pre-processing"
Jul 28 20:04:34.521857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521824    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="b0f79eb0-6692-4d9e-9152-d7bd8d922f47" containerName="payload-pre-processing"
Jul 28 20:04:34.521857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521836    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="66df0829-cddb-4d4e-958f-7f6a4a1dbead" containerName="revoke-keys"
Jul 28 20:04:34.521857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521841    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="66df0829-cddb-4d4e-958f-7f6a4a1dbead" containerName="revoke-keys"
Jul 28 20:04:34.521857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521852    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f709db3a-6ac4-4be5-b94d-f2ff059e23b7" containerName="maas-api"
Jul 28 20:04:34.521857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521857    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f709db3a-6ac4-4be5-b94d-f2ff059e23b7" containerName="maas-api"
Jul 28 20:04:34.522116 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521911    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="66df0829-cddb-4d4e-958f-7f6a4a1dbead" containerName="revoke-keys"
Jul 28 20:04:34.522116 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521918    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f709db3a-6ac4-4be5-b94d-f2ff059e23b7" containerName="maas-api"
Jul 28 20:04:34.522116 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521924    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="b0f79eb0-6692-4d9e-9152-d7bd8d922f47" containerName="payload-pre-processing"
Jul 28 20:04:34.522116 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.521930    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="40c1d1a3-09c0-458d-a160-0065c8266ccc" containerName="payload-processing"
Jul 28 20:04:34.526439 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.526419    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.528902 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.528877    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:04:34.528902 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.528889    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-2a481de7-serving-cert\""
Jul 28 20:04:34.529109 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.529095    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:04:34.529824 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.529807    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:04:34.535086 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.535067    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"]
Jul 28 20:04:34.620591 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.620550    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:34.627912 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.627883    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9jhrz\" (UniqueName: \"kubernetes.io/projected/16fce11f-e02a-448c-b8b4-789ee3e9c614-kube-api-access-9jhrz\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.628038 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.627926    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.628096 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.628075    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fce11f-e02a-448c-b8b4-789ee3e9c614-openshift-service-ca\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.728766 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.728731    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.728902 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.728837    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fce11f-e02a-448c-b8b4-789ee3e9c614-openshift-service-ca\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.728902 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.728870    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9jhrz\" (UniqueName: \"kubernetes.io/projected/16fce11f-e02a-448c-b8b4-789ee3e9c614-kube-api-access-9jhrz\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.728902 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:04:34.728886    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-serving-cert: secret "maas-api-e2e-derive-2a481de7-serving-cert" not found
Jul 28 20:04:34.729028 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:04:34.728968    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls podName:16fce11f-e02a-448c-b8b4-789ee3e9c614 nodeName:}" failed. No retries permitted until 2026-07-28 20:04:35.228951743 +0000 UTC m=+1788.040513584 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls") pod "maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp" (UID: "16fce11f-e02a-448c-b8b4-789ee3e9c614") : secret "maas-api-e2e-derive-2a481de7-serving-cert" not found
Jul 28 20:04:34.729639 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.729615    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fce11f-e02a-448c-b8b4-789ee3e9c614-openshift-service-ca\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.737281 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.737255    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9jhrz\" (UniqueName: \"kubernetes.io/projected/16fce11f-e02a-448c-b8b4-789ee3e9c614-kube-api-access-9jhrz\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:34.873589 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:34.873498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:35.190380 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.190350    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:35.232794 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.232765    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:35.235110 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.235079    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls\") pod \"maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:35.285931 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.285896    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"]
Jul 28 20:04:35.289526 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.289509    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:35.292085 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.292061    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-2a481de7-dockercfg-w2cv7\""
Jul 28 20:04:35.292216 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.292163    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-2a481de7\""
Jul 28 20:04:35.300191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.300163    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"]
Jul 28 20:04:35.316279 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.316252    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:35.319565 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.319534    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"]
Jul 28 20:04:35.322855 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.322838    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:35.333628 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.333607    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"]
Jul 28 20:04:35.434534 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.434492    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c8a1e0-4d50-473b-a217-0e204f2901c0-plugins-config-volume\") pod \"payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl\" (UID: \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\") " pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:35.434697 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.434557    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbd2s\" (UniqueName: \"kubernetes.io/projected/f4c8a1e0-4d50-473b-a217-0e204f2901c0-kube-api-access-hbd2s\") pod \"payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl\" (UID: \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\") " pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:35.434697 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.434584    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7\" (UID: \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:35.434697 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.434613    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thszs\" (UniqueName: \"kubernetes.io/projected/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-kube-api-access-thszs\") pod \"payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7\" (UID: \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:35.437349 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.437311    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:35.535890 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.535837    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c8a1e0-4d50-473b-a217-0e204f2901c0-plugins-config-volume\") pod \"payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl\" (UID: \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\") " pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:35.536263 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.535928    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbd2s\" (UniqueName: \"kubernetes.io/projected/f4c8a1e0-4d50-473b-a217-0e204f2901c0-kube-api-access-hbd2s\") pod \"payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl\" (UID: \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\") " pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:35.536263 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.535958    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7\" (UID: \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:35.536263 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.535991    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thszs\" (UniqueName: \"kubernetes.io/projected/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-kube-api-access-thszs\") pod \"payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7\" (UID: \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:35.536638 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.536615    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c8a1e0-4d50-473b-a217-0e204f2901c0-plugins-config-volume\") pod \"payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl\" (UID: \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\") " pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:35.536801 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.536780    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7\" (UID: \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:35.549468 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.549436    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thszs\" (UniqueName: \"kubernetes.io/projected/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-kube-api-access-thszs\") pod \"payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7\" (UID: \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:35.549590 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.549444    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbd2s\" (UniqueName: \"kubernetes.io/projected/f4c8a1e0-4d50-473b-a217-0e204f2901c0-kube-api-access-hbd2s\") pod \"payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl\" (UID: \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\") " pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:35.572140 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.572108    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"]
Jul 28 20:04:35.574626 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:04:35.574604    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16fce11f_e02a_448c_b8b4_789ee3e9c614.slice/crio-8d54c447dcfc7648262a14e237af666fc187eb47a4eb4029290ab3cc3547ce34 WatchSource:0}: Error finding container 8d54c447dcfc7648262a14e237af666fc187eb47a4eb4029290ab3cc3547ce34: Status 404 returned error can't find the container with id 8d54c447dcfc7648262a14e237af666fc187eb47a4eb4029290ab3cc3547ce34
Jul 28 20:04:35.601247 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.601220    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:35.633498 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.633461    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:35.747151 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.747121    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"]
Jul 28 20:04:35.749607 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:04:35.749442    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf94e7c_26ee_4fc8_9d07_00ae1dc2cfe5.slice/crio-c82edfaf614e2306b994fe4f7c7b10c2251058c4e2a73fcc369de8762baa7eb6 WatchSource:0}: Error finding container c82edfaf614e2306b994fe4f7c7b10c2251058c4e2a73fcc369de8762baa7eb6: Status 404 returned error can't find the container with id c82edfaf614e2306b994fe4f7c7b10c2251058c4e2a73fcc369de8762baa7eb6
Jul 28 20:04:35.778304 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:35.778275    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"]
Jul 28 20:04:35.780347 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:04:35.780307    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c8a1e0_4d50_473b_a217_0e204f2901c0.slice/crio-71b45c0534f013643fcfd874a985ff0d2151473678dbc4d735c9d3b433bd6e3a WatchSource:0}: Error finding container 71b45c0534f013643fcfd874a985ff0d2151473678dbc4d735c9d3b433bd6e3a: Status 404 returned error can't find the container with id 71b45c0534f013643fcfd874a985ff0d2151473678dbc4d735c9d3b433bd6e3a
Jul 28 20:04:36.228791 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.228756    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl" event={"ID":"f4c8a1e0-4d50-473b-a217-0e204f2901c0","Type":"ContainerStarted","Data":"b4350afbf13b11c628cd77245dd008057ee6346480654e451d1cdb636fdc75e2"}
Jul 28 20:04:36.228963 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.228810    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl" event={"ID":"f4c8a1e0-4d50-473b-a217-0e204f2901c0","Type":"ContainerStarted","Data":"71b45c0534f013643fcfd874a985ff0d2151473678dbc4d735c9d3b433bd6e3a"}
Jul 28 20:04:36.228963 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.228838    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:36.230257 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.230224    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7" event={"ID":"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5","Type":"ContainerStarted","Data":"1616653758a062ab27aab8e16d4c17eaaa40760f31676178aea4afc25936dc17"}
Jul 28 20:04:36.230399 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.230278    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7" event={"ID":"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5","Type":"ContainerStarted","Data":"c82edfaf614e2306b994fe4f7c7b10c2251058c4e2a73fcc369de8762baa7eb6"}
Jul 28 20:04:36.230399 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.230344    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:36.231745 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.231722    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp" event={"ID":"16fce11f-e02a-448c-b8b4-789ee3e9c614","Type":"ContainerStarted","Data":"baef0190c5b17015c718b3d5f6c490dbde6244de88dc951e0caee0a5caf19444"}
Jul 28 20:04:36.231852 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.231759    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp" event={"ID":"16fce11f-e02a-448c-b8b4-789ee3e9c614","Type":"ContainerStarted","Data":"8d54c447dcfc7648262a14e237af666fc187eb47a4eb4029290ab3cc3547ce34"}
Jul 28 20:04:36.231909 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.231873    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:36.244917 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.244876    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl" podStartSLOduration=1.244862893 podStartE2EDuration="1.244862893s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:04:36.24331728 +0000 UTC m=+1789.054879172" watchObservedRunningTime="2026-07-28 20:04:36.244862893 +0000 UTC m=+1789.056424734"
Jul 28 20:04:36.258723 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.258645    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7" podStartSLOduration=1.25863346 podStartE2EDuration="1.25863346s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:04:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:04:36.258049632 +0000 UTC m=+1789.069611496" watchObservedRunningTime="2026-07-28 20:04:36.25863346 +0000 UTC m=+1789.070195323"
Jul 28 20:04:36.274489 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:36.274448    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp" podStartSLOduration=1.854186664 podStartE2EDuration="2.274435953s" totalImagesPullingTime="420.249289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:04:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:04:36.273276147 +0000 UTC m=+1789.084838031" watchObservedRunningTime="2026-07-28 20:04:36.274435953 +0000 UTC m=+1789.085997816"
Jul 28 20:04:42.241516 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:42.241441    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:47.238204 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:47.238171    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:47.238611 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:47.238361    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:47.672304 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:47.672270    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:04:47.675921 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:47.675894    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:04:48.158025 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.157991    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"]
Jul 28 20:04:48.162941 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.162922    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"
Jul 28 20:04:48.165680 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.165656    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:04:48.167737 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.167653    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"]
Jul 28 20:04:48.253459 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.253422    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6990dc35-dbf9-4dec-8dc4-066b1dff8641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g\" (UID: \"6990dc35-dbf9-4dec-8dc4-066b1dff8641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"
Jul 28 20:04:48.354763 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.354726    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6990dc35-dbf9-4dec-8dc4-066b1dff8641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g\" (UID: \"6990dc35-dbf9-4dec-8dc4-066b1dff8641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"
Jul 28 20:04:48.355351 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.355305    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6990dc35-dbf9-4dec-8dc4-066b1dff8641-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g\" (UID: \"6990dc35-dbf9-4dec-8dc4-066b1dff8641\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"
Jul 28 20:04:48.475139 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.475046    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"
Jul 28 20:04:48.815776 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:48.815694    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"]
Jul 28 20:04:48.818882 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:04:48.818855    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6990dc35_dbf9_4dec_8dc4_066b1dff8641.slice/crio-a514d3671b8c4b3a097abb131cec67d73ec143cae1c4069ddc4d825907a55882 WatchSource:0}: Error finding container a514d3671b8c4b3a097abb131cec67d73ec143cae1c4069ddc4d825907a55882: Status 404 returned error can't find the container with id a514d3671b8c4b3a097abb131cec67d73ec143cae1c4069ddc4d825907a55882
Jul 28 20:04:49.300711 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:49.300676    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g" event={"ID":"6990dc35-dbf9-4dec-8dc4-066b1dff8641","Type":"ContainerStarted","Data":"673bc8b3da3c6442318248961561499a429e65ba6bb3be36e264888cf3ca68d0"}
Jul 28 20:04:49.301096 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:49.300726    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g" event={"ID":"6990dc35-dbf9-4dec-8dc4-066b1dff8641","Type":"ContainerStarted","Data":"a514d3671b8c4b3a097abb131cec67d73ec143cae1c4069ddc4d825907a55882"}
Jul 28 20:04:49.315069 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:49.315023    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g" podStartSLOduration=1.315008614 podStartE2EDuration="1.315008614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:04:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:04:49.314880097 +0000 UTC m=+1802.126441960" watchObservedRunningTime="2026-07-28 20:04:49.315008614 +0000 UTC m=+1802.126570478"
Jul 28 20:04:50.307202 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:50.307162    2568 generic.go:356] "Generic (PLEG): container finished" podID="6990dc35-dbf9-4dec-8dc4-066b1dff8641" containerID="673bc8b3da3c6442318248961561499a429e65ba6bb3be36e264888cf3ca68d0" exitCode=0
Jul 28 20:04:50.307666 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:50.307248    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g" event={"ID":"6990dc35-dbf9-4dec-8dc4-066b1dff8641","Type":"ContainerDied","Data":"673bc8b3da3c6442318248961561499a429e65ba6bb3be36e264888cf3ca68d0"}
Jul 28 20:04:51.445615 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:51.445589    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"
Jul 28 20:04:51.582954 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:51.582865    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6990dc35-dbf9-4dec-8dc4-066b1dff8641-maas-api-service-ca\") pod \"6990dc35-dbf9-4dec-8dc4-066b1dff8641\" (UID: \"6990dc35-dbf9-4dec-8dc4-066b1dff8641\") "
Jul 28 20:04:51.583212 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:51.583187    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6990dc35-dbf9-4dec-8dc4-066b1dff8641-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6990dc35-dbf9-4dec-8dc4-066b1dff8641" (UID: "6990dc35-dbf9-4dec-8dc4-066b1dff8641"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:51.683767 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:51.683715    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6990dc35-dbf9-4dec-8dc4-066b1dff8641-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:52.318498 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:52.318468    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"
Jul 28 20:04:52.318722 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:52.318471    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g" event={"ID":"6990dc35-dbf9-4dec-8dc4-066b1dff8641","Type":"ContainerDied","Data":"a514d3671b8c4b3a097abb131cec67d73ec143cae1c4069ddc4d825907a55882"}
Jul 28 20:04:52.318722 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:52.318596    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a514d3671b8c4b3a097abb131cec67d73ec143cae1c4069ddc4d825907a55882"
Jul 28 20:04:54.135829 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.135793    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"]
Jul 28 20:04:54.136352 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.136048    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp" podUID="16fce11f-e02a-448c-b8b4-789ee3e9c614" containerName="maas-api" containerID="cri-o://baef0190c5b17015c718b3d5f6c490dbde6244de88dc951e0caee0a5caf19444" gracePeriod=30
Jul 28 20:04:54.172094 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.172057    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:54.193102 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.193076    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"]
Jul 28 20:04:54.193308 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.193278    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl" podUID="f4c8a1e0-4d50-473b-a217-0e204f2901c0" containerName="payload-processing" containerID="cri-o://b4350afbf13b11c628cd77245dd008057ee6346480654e451d1cdb636fdc75e2" gracePeriod=30
Jul 28 20:04:54.199426 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.199400    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"]
Jul 28 20:04:54.199636 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.199611    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7" podUID="faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5" containerName="payload-pre-processing" containerID="cri-o://1616653758a062ab27aab8e16d4c17eaaa40760f31676178aea4afc25936dc17" gracePeriod=30
Jul 28 20:04:54.272041 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.272013    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:54.331219 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.331179    2568 generic.go:356] "Generic (PLEG): container finished" podID="faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5" containerID="1616653758a062ab27aab8e16d4c17eaaa40760f31676178aea4afc25936dc17" exitCode=0
Jul 28 20:04:54.331403 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.331218    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7" event={"ID":"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5","Type":"ContainerDied","Data":"1616653758a062ab27aab8e16d4c17eaaa40760f31676178aea4afc25936dc17"}
Jul 28 20:04:54.332923 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.332900    2568 generic.go:356] "Generic (PLEG): container finished" podID="16fce11f-e02a-448c-b8b4-789ee3e9c614" containerID="baef0190c5b17015c718b3d5f6c490dbde6244de88dc951e0caee0a5caf19444" exitCode=0
Jul 28 20:04:54.333040 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.332975    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp" event={"ID":"16fce11f-e02a-448c-b8b4-789ee3e9c614","Type":"ContainerDied","Data":"baef0190c5b17015c718b3d5f6c490dbde6244de88dc951e0caee0a5caf19444"}
Jul 28 20:04:54.334703 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.334680    2568 generic.go:356] "Generic (PLEG): container finished" podID="f4c8a1e0-4d50-473b-a217-0e204f2901c0" containerID="b4350afbf13b11c628cd77245dd008057ee6346480654e451d1cdb636fdc75e2" exitCode=0
Jul 28 20:04:54.334806 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.334748    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl" event={"ID":"f4c8a1e0-4d50-473b-a217-0e204f2901c0","Type":"ContainerDied","Data":"b4350afbf13b11c628cd77245dd008057ee6346480654e451d1cdb636fdc75e2"}
Jul 28 20:04:54.373697 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.373608    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:54.473546 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.473507    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:54.499473 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.499447    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:54.502667 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.502644    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:54.506784 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.506763    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:54.508407 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.508385    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"]
Jul 28 20:04:54.510200 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.510182    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-2a481de7-wcz4g"]
Jul 28 20:04:54.611526 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.611490    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls\") pod \"16fce11f-e02a-448c-b8b4-789ee3e9c614\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") "
Jul 28 20:04:54.611696 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.611538    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9jhrz\" (UniqueName: \"kubernetes.io/projected/16fce11f-e02a-448c-b8b4-789ee3e9c614-kube-api-access-9jhrz\") pod \"16fce11f-e02a-448c-b8b4-789ee3e9c614\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") "
Jul 28 20:04:54.611696 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.611559    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c8a1e0-4d50-473b-a217-0e204f2901c0-plugins-config-volume\") pod \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\" (UID: \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\") "
Jul 28 20:04:54.611696 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.611629    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thszs\" (UniqueName: \"kubernetes.io/projected/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-kube-api-access-thszs\") pod \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\" (UID: \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\") "
Jul 28 20:04:54.611696 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.611678    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-plugins-config-volume\") pod \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\" (UID: \"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5\") "
Jul 28 20:04:54.611942 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.611730    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fce11f-e02a-448c-b8b4-789ee3e9c614-openshift-service-ca\") pod \"16fce11f-e02a-448c-b8b4-789ee3e9c614\" (UID: \"16fce11f-e02a-448c-b8b4-789ee3e9c614\") "
Jul 28 20:04:54.611942 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.611760    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbd2s\" (UniqueName: \"kubernetes.io/projected/f4c8a1e0-4d50-473b-a217-0e204f2901c0-kube-api-access-hbd2s\") pod \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\" (UID: \"f4c8a1e0-4d50-473b-a217-0e204f2901c0\") "
Jul 28 20:04:54.612049 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.612016    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4c8a1e0-4d50-473b-a217-0e204f2901c0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f4c8a1e0-4d50-473b-a217-0e204f2901c0" (UID: "f4c8a1e0-4d50-473b-a217-0e204f2901c0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:54.612116 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.612080    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5" (UID: "faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:54.612175 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.612134    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16fce11f-e02a-448c-b8b4-789ee3e9c614-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16fce11f-e02a-448c-b8b4-789ee3e9c614" (UID: "16fce11f-e02a-448c-b8b4-789ee3e9c614"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:04:54.613922 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.613899    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16fce11f-e02a-448c-b8b4-789ee3e9c614-kube-api-access-9jhrz" (OuterVolumeSpecName: "kube-api-access-9jhrz") pod "16fce11f-e02a-448c-b8b4-789ee3e9c614" (UID: "16fce11f-e02a-448c-b8b4-789ee3e9c614"). InnerVolumeSpecName "kube-api-access-9jhrz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:04:54.613922 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.613912    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f4c8a1e0-4d50-473b-a217-0e204f2901c0-kube-api-access-hbd2s" (OuterVolumeSpecName: "kube-api-access-hbd2s") pod "f4c8a1e0-4d50-473b-a217-0e204f2901c0" (UID: "f4c8a1e0-4d50-473b-a217-0e204f2901c0"). InnerVolumeSpecName "kube-api-access-hbd2s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:04:54.614076 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.613912    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-kube-api-access-thszs" (OuterVolumeSpecName: "kube-api-access-thszs") pod "faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5" (UID: "faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5"). InnerVolumeSpecName "kube-api-access-thszs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:04:54.614076 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.613899    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16fce11f-e02a-448c-b8b4-789ee3e9c614" (UID: "16fce11f-e02a-448c-b8b4-789ee3e9c614"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:04:54.655920 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.655891    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:54.712913 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.712838    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thszs\" (UniqueName: \"kubernetes.io/projected/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-kube-api-access-thszs\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:54.712913 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.712865    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:54.712913 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.712875    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16fce11f-e02a-448c-b8b4-789ee3e9c614-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:54.712913 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.712884    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbd2s\" (UniqueName: \"kubernetes.io/projected/f4c8a1e0-4d50-473b-a217-0e204f2901c0-kube-api-access-hbd2s\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:54.712913 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.712893    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16fce11f-e02a-448c-b8b4-789ee3e9c614-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:54.712913 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.712902    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9jhrz\" (UniqueName: \"kubernetes.io/projected/16fce11f-e02a-448c-b8b4-789ee3e9c614-kube-api-access-9jhrz\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:54.712913 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.712911    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f4c8a1e0-4d50-473b-a217-0e204f2901c0-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:04:54.915954 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:54.915918    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:55.341210 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.341176    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"
Jul 28 20:04:55.341689 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.341182    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7" event={"ID":"faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5","Type":"ContainerDied","Data":"c82edfaf614e2306b994fe4f7c7b10c2251058c4e2a73fcc369de8762baa7eb6"}
Jul 28 20:04:55.341689 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.341332    2568 scope.go:117] "RemoveContainer" containerID="1616653758a062ab27aab8e16d4c17eaaa40760f31676178aea4afc25936dc17"
Jul 28 20:04:55.342711 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.342629    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp" event={"ID":"16fce11f-e02a-448c-b8b4-789ee3e9c614","Type":"ContainerDied","Data":"8d54c447dcfc7648262a14e237af666fc187eb47a4eb4029290ab3cc3547ce34"}
Jul 28 20:04:55.342711 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.342634    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"
Jul 28 20:04:55.344310 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.344260    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"
Jul 28 20:04:55.344310 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.344278    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl" event={"ID":"f4c8a1e0-4d50-473b-a217-0e204f2901c0","Type":"ContainerDied","Data":"71b45c0534f013643fcfd874a985ff0d2151473678dbc4d735c9d3b433bd6e3a"}
Jul 28 20:04:55.356989 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.356953    2568 scope.go:117] "RemoveContainer" containerID="baef0190c5b17015c718b3d5f6c490dbde6244de88dc951e0caee0a5caf19444"
Jul 28 20:04:55.366186 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.366166    2568 scope.go:117] "RemoveContainer" containerID="b4350afbf13b11c628cd77245dd008057ee6346480654e451d1cdb636fdc75e2"
Jul 28 20:04:55.377937 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.377913    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"]
Jul 28 20:04:55.388802 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.388781    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-2a481de7-79b966864d-5szv7"]
Jul 28 20:04:55.409255 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.409194    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"]
Jul 28 20:04:55.409737 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.409716    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:55.414109 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.414085    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-2a481de7-f6b48c788-mb2mp"]
Jul 28 20:04:55.430076 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.430054    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"]
Jul 28 20:04:55.432405 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.432386    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-2a481de7-bd6c76d6b-4dhsl"]
Jul 28 20:04:55.509915 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.509886    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:55.624006 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.623958    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:04:55.677369 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.677256    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16fce11f-e02a-448c-b8b4-789ee3e9c614" path="/var/lib/kubelet/pods/16fce11f-e02a-448c-b8b4-789ee3e9c614/volumes"
Jul 28 20:04:55.677714 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.677691    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6990dc35-dbf9-4dec-8dc4-066b1dff8641" path="/var/lib/kubelet/pods/6990dc35-dbf9-4dec-8dc4-066b1dff8641/volumes"
Jul 28 20:04:55.678145 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.678128    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f4c8a1e0-4d50-473b-a217-0e204f2901c0" path="/var/lib/kubelet/pods/f4c8a1e0-4d50-473b-a217-0e204f2901c0/volumes"
Jul 28 20:04:55.678485 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:55.678471    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5" path="/var/lib/kubelet/pods/faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5/volumes"
Jul 28 20:04:57.098950 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:04:57.098909    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:03.503088 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:03.503045    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:03.614860 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:03.614824    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:03.806620 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:03.806533    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:03.906505 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:03.906470    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:04.264344 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:04.264292    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:18.585593 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:18.585549    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:30.308582 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:30.308546    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:36.736783 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:36.736747    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:36.875617 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:36.875581    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:37.018503 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:37.018423    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:52.926235 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:52.926198    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:58.583947 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:58.583907    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:58.648258 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:58.648222    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:58.763897 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:58.763850    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:05:58.859648 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:05:58.859561    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:09.349069 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:09.349037    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:10.675138 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:10.675052    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:16.535460 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:16.535408    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:16.643772 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:16.643734    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:16.735305 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:16.735273    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:16.837387 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:16.837295    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:28.361595 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:28.361560    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:28.461217 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:28.461180    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:28.561709 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:28.561671    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:30.393020 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:30.392991    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:36.450504 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:36.450470    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:36.559312 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:36.559271    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:37.190306 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:37.190273    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:43.108507 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:43.108464    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:43.213510 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:43.213474    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:43.402334 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:43.402295    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:43.720862 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:43.720787    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:43.951697 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:43.951662    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:46.726222 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.726192    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:46.816983 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.816944    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"]
Jul 28 20:06:46.817518 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817501    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16fce11f-e02a-448c-b8b4-789ee3e9c614" containerName="maas-api"
Jul 28 20:06:46.817580 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817521    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="16fce11f-e02a-448c-b8b4-789ee3e9c614" containerName="maas-api"
Jul 28 20:06:46.817580 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817535    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6990dc35-dbf9-4dec-8dc4-066b1dff8641" containerName="revoke-keys"
Jul 28 20:06:46.817580 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817544    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="6990dc35-dbf9-4dec-8dc4-066b1dff8641" containerName="revoke-keys"
Jul 28 20:06:46.817580 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817564    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4c8a1e0-4d50-473b-a217-0e204f2901c0" containerName="payload-processing"
Jul 28 20:06:46.817580 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817573    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4c8a1e0-4d50-473b-a217-0e204f2901c0" containerName="payload-processing"
Jul 28 20:06:46.817760 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817608    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5" containerName="payload-pre-processing"
Jul 28 20:06:46.817760 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817618    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5" containerName="payload-pre-processing"
Jul 28 20:06:46.817760 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817701    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="16fce11f-e02a-448c-b8b4-789ee3e9c614" containerName="maas-api"
Jul 28 20:06:46.817760 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817726    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4c8a1e0-4d50-473b-a217-0e204f2901c0" containerName="payload-processing"
Jul 28 20:06:46.817760 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817737    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="faf94e7c-26ee-4fc8-9d07-00ae1dc2cfe5" containerName="payload-pre-processing"
Jul 28 20:06:46.817760 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.817746    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="6990dc35-dbf9-4dec-8dc4-066b1dff8641" containerName="revoke-keys"
Jul 28 20:06:46.821244 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.821220    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:46.821505 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.821477    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:46.823996 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.823975    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:06:46.824207 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.824003    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:06:46.824301 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.824003    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-e9667e4b-serving-cert\""
Jul 28 20:06:46.824963 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.824945    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:06:46.830677 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.830655    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"]
Jul 28 20:06:46.890587 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.890549    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dr5vx\" (UniqueName: \"kubernetes.io/projected/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-kube-api-access-dr5vx\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:46.890759 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.890639    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:46.890759 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.890675    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-openshift-service-ca\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:46.892903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.892872    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:46.991677 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.991593    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dr5vx\" (UniqueName: \"kubernetes.io/projected/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-kube-api-access-dr5vx\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:46.991677 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.991640    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:46.991677 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.991661    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-openshift-service-ca\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:46.991949 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:06:46.991761    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-serving-cert: secret "maas-api-e2e-mt-e9667e4b-serving-cert" not found
Jul 28 20:06:46.991949 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:06:46.991873    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls podName:c2ef8aab-0827-4cd4-8c81-5f024a6bb06f nodeName:}" failed. No retries permitted until 2026-07-28 20:06:47.491850175 +0000 UTC m=+1920.303412045 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls") pod "maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw" (UID: "c2ef8aab-0827-4cd4-8c81-5f024a6bb06f") : secret "maas-api-e2e-mt-e9667e4b-serving-cert" not found
Jul 28 20:06:46.992412 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:46.992394    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-openshift-service-ca\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:47.004144 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.004119    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dr5vx\" (UniqueName: \"kubernetes.io/projected/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-kube-api-access-dr5vx\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:47.069970 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.069947    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:47.496038 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.495994    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:47.498372 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.498346    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls\") pod \"maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:47.682359 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.682314    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:47.735590 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.735563    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:06:47.743750 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.743730    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:47.786003 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.785972    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:47.804258 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.804228    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"]
Jul 28 20:06:47.808601 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.808574    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:47.808710 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.808585    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"]
Jul 28 20:06:47.811617 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.811398    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-e9667e4b\""
Jul 28 20:06:47.812681 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.812662    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:47.815310 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.815289    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-e9667e4b-dockercfg-szfg4\""
Jul 28 20:06:47.820171 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.820130    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"]
Jul 28 20:06:47.822810 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.822789    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"]
Jul 28 20:06:47.889284 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.889244    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"]
Jul 28 20:06:47.892143 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:06:47.892117    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2ef8aab_0827_4cd4_8c81_5f024a6bb06f.slice/crio-f707fbc5f909382b66bc51a6d6f8d1085d25d31b42ef989438d1660bc6fdacdd WatchSource:0}: Error finding container f707fbc5f909382b66bc51a6d6f8d1085d25d31b42ef989438d1660bc6fdacdd: Status 404 returned error can't find the container with id f707fbc5f909382b66bc51a6d6f8d1085d25d31b42ef989438d1660bc6fdacdd
Jul 28 20:06:47.893912 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.893890    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:06:47.900506 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.900483    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2757f\" (UniqueName: \"kubernetes.io/projected/d466a873-99d7-46f8-ac34-59dbc47d1137-kube-api-access-2757f\") pod \"payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg\" (UID: \"d466a873-99d7-46f8-ac34-59dbc47d1137\") " pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:47.900627 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.900597    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d466a873-99d7-46f8-ac34-59dbc47d1137-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg\" (UID: \"d466a873-99d7-46f8-ac34-59dbc47d1137\") " pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:47.900695 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.900642    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-62hkj\" (UniqueName: \"kubernetes.io/projected/81b2217e-21c7-42ed-b787-acb9131f78b0-kube-api-access-62hkj\") pod \"payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q\" (UID: \"81b2217e-21c7-42ed-b787-acb9131f78b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:47.901067 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.900729    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81b2217e-21c7-42ed-b787-acb9131f78b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q\" (UID: \"81b2217e-21c7-42ed-b787-acb9131f78b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:47.901067 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:47.900733    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw" event={"ID":"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f","Type":"ContainerStarted","Data":"f707fbc5f909382b66bc51a6d6f8d1085d25d31b42ef989438d1660bc6fdacdd"}
Jul 28 20:06:48.001926 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.001838    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81b2217e-21c7-42ed-b787-acb9131f78b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q\" (UID: \"81b2217e-21c7-42ed-b787-acb9131f78b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:48.001926 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.001883    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2757f\" (UniqueName: \"kubernetes.io/projected/d466a873-99d7-46f8-ac34-59dbc47d1137-kube-api-access-2757f\") pod \"payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg\" (UID: \"d466a873-99d7-46f8-ac34-59dbc47d1137\") " pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:48.002163 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.001960    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d466a873-99d7-46f8-ac34-59dbc47d1137-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg\" (UID: \"d466a873-99d7-46f8-ac34-59dbc47d1137\") " pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:48.002163 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.001991    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-62hkj\" (UniqueName: \"kubernetes.io/projected/81b2217e-21c7-42ed-b787-acb9131f78b0-kube-api-access-62hkj\") pod \"payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q\" (UID: \"81b2217e-21c7-42ed-b787-acb9131f78b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:48.002621 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.002598    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81b2217e-21c7-42ed-b787-acb9131f78b0-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q\" (UID: \"81b2217e-21c7-42ed-b787-acb9131f78b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:48.002711 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.002692    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d466a873-99d7-46f8-ac34-59dbc47d1137-plugins-config-volume\") pod \"payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg\" (UID: \"d466a873-99d7-46f8-ac34-59dbc47d1137\") " pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:48.010089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.010058    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2757f\" (UniqueName: \"kubernetes.io/projected/d466a873-99d7-46f8-ac34-59dbc47d1137-kube-api-access-2757f\") pod \"payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg\" (UID: \"d466a873-99d7-46f8-ac34-59dbc47d1137\") " pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:48.010243 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.010222    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-62hkj\" (UniqueName: \"kubernetes.io/projected/81b2217e-21c7-42ed-b787-acb9131f78b0-kube-api-access-62hkj\") pod \"payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q\" (UID: \"81b2217e-21c7-42ed-b787-acb9131f78b0\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:48.125569 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.125537    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:48.132518 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.132494    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:48.296488 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.296460    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"]
Jul 28 20:06:48.298985 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:06:48.298960    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd466a873_99d7_46f8_ac34_59dbc47d1137.slice/crio-445bdd22dd2641abcd2ad749c719495de90a2100d5700aa1740c7406bb3a7b69 WatchSource:0}: Error finding container 445bdd22dd2641abcd2ad749c719495de90a2100d5700aa1740c7406bb3a7b69: Status 404 returned error can't find the container with id 445bdd22dd2641abcd2ad749c719495de90a2100d5700aa1740c7406bb3a7b69
Jul 28 20:06:48.475378 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.475348    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"]
Jul 28 20:06:48.477709 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:06:48.477673    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81b2217e_21c7_42ed_b787_acb9131f78b0.slice/crio-ad2ca53fbb78d10efde8892bb326b27f77d25353c22dd0cd8084630f6ccbd416 WatchSource:0}: Error finding container ad2ca53fbb78d10efde8892bb326b27f77d25353c22dd0cd8084630f6ccbd416: Status 404 returned error can't find the container with id ad2ca53fbb78d10efde8892bb326b27f77d25353c22dd0cd8084630f6ccbd416
Jul 28 20:06:48.907677 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.907635    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q" event={"ID":"81b2217e-21c7-42ed-b787-acb9131f78b0","Type":"ContainerStarted","Data":"99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1"}
Jul 28 20:06:48.908156 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.907696    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:06:48.908156 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.907722    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q" event={"ID":"81b2217e-21c7-42ed-b787-acb9131f78b0","Type":"ContainerStarted","Data":"ad2ca53fbb78d10efde8892bb326b27f77d25353c22dd0cd8084630f6ccbd416"}
Jul 28 20:06:48.909255 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.909224    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw" event={"ID":"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f","Type":"ContainerStarted","Data":"ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32"}
Jul 28 20:06:48.909415 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.909363    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:48.910758 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.910731    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg" event={"ID":"d466a873-99d7-46f8-ac34-59dbc47d1137","Type":"ContainerStarted","Data":"1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88"}
Jul 28 20:06:48.910878 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.910779    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg" event={"ID":"d466a873-99d7-46f8-ac34-59dbc47d1137","Type":"ContainerStarted","Data":"445bdd22dd2641abcd2ad749c719495de90a2100d5700aa1740c7406bb3a7b69"}
Jul 28 20:06:48.910878 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.910815    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:48.928172 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.928129    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q" podStartSLOduration=1.9281164469999998 podStartE2EDuration="1.928116447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:06:48.925714831 +0000 UTC m=+1921.737276700" watchObservedRunningTime="2026-07-28 20:06:48.928116447 +0000 UTC m=+1921.739678307"
Jul 28 20:06:48.944670 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.944630    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg" podStartSLOduration=1.9446174520000001 podStartE2EDuration="1.944617452s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:06:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:06:48.942868803 +0000 UTC m=+1921.754430666" watchObservedRunningTime="2026-07-28 20:06:48.944617452 +0000 UTC m=+1921.756179314"
Jul 28 20:06:48.958388 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:48.958351    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw" podStartSLOduration=2.525303722 podStartE2EDuration="2.958312538s" totalImagesPullingTime="433.008816ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:06:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:06:48.958163758 +0000 UTC m=+1921.769725621" watchObservedRunningTime="2026-07-28 20:06:48.958312538 +0000 UTC m=+1921.769874402"
Jul 28 20:06:49.686227 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:49.686186    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:49.786409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:49.786373    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:06:54.922014 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:54.921984    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:06:59.918565 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:59.918530    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:06:59.919073 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:06:59.918705    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:07:11.747806 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:11.747771    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:12.607049 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:12.607009    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:12.708110 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:12.708071    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:12.939158 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:12.939123    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:18.696921 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:18.696884    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:18.772063 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:18.772031    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:19.076517 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.076431    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"]
Jul 28 20:07:19.080814 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.080797    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"
Jul 28 20:07:19.083311 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.083286    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:07:19.087050 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.087026    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"]
Jul 28 20:07:19.186600 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.186550    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e512053-31ca-48a4-83bb-1700f48f8554-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f\" (UID: \"1e512053-31ca-48a4-83bb-1700f48f8554\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"
Jul 28 20:07:19.287903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.287863    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e512053-31ca-48a4-83bb-1700f48f8554-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f\" (UID: \"1e512053-31ca-48a4-83bb-1700f48f8554\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"
Jul 28 20:07:19.288567 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.288546    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e512053-31ca-48a4-83bb-1700f48f8554-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f\" (UID: \"1e512053-31ca-48a4-83bb-1700f48f8554\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"
Jul 28 20:07:19.394489 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.394445    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"
Jul 28 20:07:19.533202 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:19.533149    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"]
Jul 28 20:07:19.535721 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:07:19.535684    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e512053_31ca_48a4_83bb_1700f48f8554.slice/crio-ab7d0937bc4b1f2775522eb59a3f979e07a2875168f2f6ef8a6c705211bc1ff0 WatchSource:0}: Error finding container ab7d0937bc4b1f2775522eb59a3f979e07a2875168f2f6ef8a6c705211bc1ff0: Status 404 returned error can't find the container with id ab7d0937bc4b1f2775522eb59a3f979e07a2875168f2f6ef8a6c705211bc1ff0
Jul 28 20:07:20.073903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:20.073810    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f" event={"ID":"1e512053-31ca-48a4-83bb-1700f48f8554","Type":"ContainerStarted","Data":"70b9387444a782e0188cb5e50cee6ef2463f25a168522ef52cb5b5ed83c99aa3"}
Jul 28 20:07:20.073903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:20.073864    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f" event={"ID":"1e512053-31ca-48a4-83bb-1700f48f8554","Type":"ContainerStarted","Data":"ab7d0937bc4b1f2775522eb59a3f979e07a2875168f2f6ef8a6c705211bc1ff0"}
Jul 28 20:07:20.114280 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:20.114232    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f" podStartSLOduration=1.114217716 podStartE2EDuration="1.114217716s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:07:20.110615452 +0000 UTC m=+1952.922177315" watchObservedRunningTime="2026-07-28 20:07:20.114217716 +0000 UTC m=+1952.925779579"
Jul 28 20:07:21.079663 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:21.079627    2568 generic.go:356] "Generic (PLEG): container finished" podID="1e512053-31ca-48a4-83bb-1700f48f8554" containerID="70b9387444a782e0188cb5e50cee6ef2463f25a168522ef52cb5b5ed83c99aa3" exitCode=0
Jul 28 20:07:21.080062 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:21.079723    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f" event={"ID":"1e512053-31ca-48a4-83bb-1700f48f8554","Type":"ContainerDied","Data":"70b9387444a782e0188cb5e50cee6ef2463f25a168522ef52cb5b5ed83c99aa3"}
Jul 28 20:07:22.245810 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:22.245784    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"
Jul 28 20:07:22.314824 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:22.314792    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e512053-31ca-48a4-83bb-1700f48f8554-maas-api-service-ca\") pod \"1e512053-31ca-48a4-83bb-1700f48f8554\" (UID: \"1e512053-31ca-48a4-83bb-1700f48f8554\") "
Jul 28 20:07:22.315146 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:22.315124    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e512053-31ca-48a4-83bb-1700f48f8554-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e512053-31ca-48a4-83bb-1700f48f8554" (UID: "1e512053-31ca-48a4-83bb-1700f48f8554"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:07:22.415671 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:22.415624    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e512053-31ca-48a4-83bb-1700f48f8554-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:07:23.090882 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:23.090850    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"
Jul 28 20:07:23.091062 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:23.090887    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f" event={"ID":"1e512053-31ca-48a4-83bb-1700f48f8554","Type":"ContainerDied","Data":"ab7d0937bc4b1f2775522eb59a3f979e07a2875168f2f6ef8a6c705211bc1ff0"}
Jul 28 20:07:23.091062 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:23.090933    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ab7d0937bc4b1f2775522eb59a3f979e07a2875168f2f6ef8a6c705211bc1ff0"
Jul 28 20:07:29.687570 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.687532    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"]
Jul 28 20:07:29.688355 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.688284    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw" podUID="c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" containerName="maas-api" containerID="cri-o://ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32" gracePeriod=30
Jul 28 20:07:29.711473 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.711449    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:29.739055 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.739022    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"]
Jul 28 20:07:29.739298 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.739272    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg" podUID="d466a873-99d7-46f8-ac34-59dbc47d1137" containerName="payload-processing" containerID="cri-o://1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88" gracePeriod=30
Jul 28 20:07:29.748065 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.748037    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"]
Jul 28 20:07:29.748272 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.748250    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q" podUID="81b2217e-21c7-42ed-b787-acb9131f78b0" containerName="payload-pre-processing" containerID="cri-o://99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1" gracePeriod=30
Jul 28 20:07:29.780759 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.780729    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:29.850522 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.850493    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:29.895067 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.895026    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"]
Jul 28 20:07:29.896888 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.896859    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9667e4b-l5p4f"]
Jul 28 20:07:29.912240 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.912210    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:29.917062 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.916967    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q" podUID="81b2217e-21c7-42ed-b787-acb9131f78b0" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.55:9004: connect: connection refused"
Jul 28 20:07:29.917350 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:29.917298    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw" podUID="c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.54:8443/health\": dial tcp 10.132.0.54:8443: connect: connection refused"
Jul 28 20:07:30.034816 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.034788    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:07:30.053629 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.053603    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:07:30.057583 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.057552    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:07:30.072368 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.072298    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-openshift-service-ca\") pod \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") "
Jul 28 20:07:30.072368 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.072356    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81b2217e-21c7-42ed-b787-acb9131f78b0-plugins-config-volume\") pod \"81b2217e-21c7-42ed-b787-acb9131f78b0\" (UID: \"81b2217e-21c7-42ed-b787-acb9131f78b0\") "
Jul 28 20:07:30.074683 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.072409    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2757f\" (UniqueName: \"kubernetes.io/projected/d466a873-99d7-46f8-ac34-59dbc47d1137-kube-api-access-2757f\") pod \"d466a873-99d7-46f8-ac34-59dbc47d1137\" (UID: \"d466a873-99d7-46f8-ac34-59dbc47d1137\") "
Jul 28 20:07:30.074683 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.072434    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-62hkj\" (UniqueName: \"kubernetes.io/projected/81b2217e-21c7-42ed-b787-acb9131f78b0-kube-api-access-62hkj\") pod \"81b2217e-21c7-42ed-b787-acb9131f78b0\" (UID: \"81b2217e-21c7-42ed-b787-acb9131f78b0\") "
Jul 28 20:07:30.074683 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.072488    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls\") pod \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") "
Jul 28 20:07:30.074683 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.072518    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d466a873-99d7-46f8-ac34-59dbc47d1137-plugins-config-volume\") pod \"d466a873-99d7-46f8-ac34-59dbc47d1137\" (UID: \"d466a873-99d7-46f8-ac34-59dbc47d1137\") "
Jul 28 20:07:30.074683 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.072559    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dr5vx\" (UniqueName: \"kubernetes.io/projected/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-kube-api-access-dr5vx\") pod \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\" (UID: \"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f\") "
Jul 28 20:07:30.075041 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.074679    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" (UID: "c2ef8aab-0827-4cd4-8c81-5f024a6bb06f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:07:30.075105 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.075071    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d466a873-99d7-46f8-ac34-59dbc47d1137-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "d466a873-99d7-46f8-ac34-59dbc47d1137" (UID: "d466a873-99d7-46f8-ac34-59dbc47d1137"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:07:30.075284 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.075250    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81b2217e-21c7-42ed-b787-acb9131f78b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "81b2217e-21c7-42ed-b787-acb9131f78b0" (UID: "81b2217e-21c7-42ed-b787-acb9131f78b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:07:30.076490 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.076399    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-kube-api-access-dr5vx" (OuterVolumeSpecName: "kube-api-access-dr5vx") pod "c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" (UID: "c2ef8aab-0827-4cd4-8c81-5f024a6bb06f"). InnerVolumeSpecName "kube-api-access-dr5vx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:07:30.076490 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.076485    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d466a873-99d7-46f8-ac34-59dbc47d1137-kube-api-access-2757f" (OuterVolumeSpecName: "kube-api-access-2757f") pod "d466a873-99d7-46f8-ac34-59dbc47d1137" (UID: "d466a873-99d7-46f8-ac34-59dbc47d1137"). InnerVolumeSpecName "kube-api-access-2757f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:07:30.077473 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.077449    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" (UID: "c2ef8aab-0827-4cd4-8c81-5f024a6bb06f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:07:30.077630 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.077557    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81b2217e-21c7-42ed-b787-acb9131f78b0-kube-api-access-62hkj" (OuterVolumeSpecName: "kube-api-access-62hkj") pod "81b2217e-21c7-42ed-b787-acb9131f78b0" (UID: "81b2217e-21c7-42ed-b787-acb9131f78b0"). InnerVolumeSpecName "kube-api-access-62hkj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:07:30.126231 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.126200    2568 generic.go:356] "Generic (PLEG): container finished" podID="d466a873-99d7-46f8-ac34-59dbc47d1137" containerID="1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88" exitCode=0
Jul 28 20:07:30.126412 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.126256    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"
Jul 28 20:07:30.126412 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.126292    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg" event={"ID":"d466a873-99d7-46f8-ac34-59dbc47d1137","Type":"ContainerDied","Data":"1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88"}
Jul 28 20:07:30.126412 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.126373    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg" event={"ID":"d466a873-99d7-46f8-ac34-59dbc47d1137","Type":"ContainerDied","Data":"445bdd22dd2641abcd2ad749c719495de90a2100d5700aa1740c7406bb3a7b69"}
Jul 28 20:07:30.126412 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.126406    2568 scope.go:117] "RemoveContainer" containerID="1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88"
Jul 28 20:07:30.127673 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.127651    2568 generic.go:356] "Generic (PLEG): container finished" podID="81b2217e-21c7-42ed-b787-acb9131f78b0" containerID="99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1" exitCode=0
Jul 28 20:07:30.127804 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.127688    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"
Jul 28 20:07:30.127804 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.127737    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q" event={"ID":"81b2217e-21c7-42ed-b787-acb9131f78b0","Type":"ContainerDied","Data":"99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1"}
Jul 28 20:07:30.127804 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.127788    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q" event={"ID":"81b2217e-21c7-42ed-b787-acb9131f78b0","Type":"ContainerDied","Data":"ad2ca53fbb78d10efde8892bb326b27f77d25353c22dd0cd8084630f6ccbd416"}
Jul 28 20:07:30.129091 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.129067    2568 generic.go:356] "Generic (PLEG): container finished" podID="c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" containerID="ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32" exitCode=0
Jul 28 20:07:30.129207 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.129107    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"
Jul 28 20:07:30.129207 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.129163    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw" event={"ID":"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f","Type":"ContainerDied","Data":"ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32"}
Jul 28 20:07:30.129390 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.129215    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw" event={"ID":"c2ef8aab-0827-4cd4-8c81-5f024a6bb06f","Type":"ContainerDied","Data":"f707fbc5f909382b66bc51a6d6f8d1085d25d31b42ef989438d1660bc6fdacdd"}
Jul 28 20:07:30.137219 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.137098    2568 scope.go:117] "RemoveContainer" containerID="1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88"
Jul 28 20:07:30.137627 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:07:30.137599    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88\": container with ID starting with 1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88 not found: ID does not exist" containerID="1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88"
Jul 28 20:07:30.137711 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.137637    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88"} err="failed to get container status \"1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88\": rpc error: code = NotFound desc = could not find container \"1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88\": container with ID starting with 1f8b7bb24159c9f4c41d1d2d59bd886d42caf8cf820fffd51c9d90c1b32dca88 not found: ID does not exist"
Jul 28 20:07:30.137711 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.137658    2568 scope.go:117] "RemoveContainer" containerID="99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1"
Jul 28 20:07:30.146199 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.146181    2568 scope.go:117] "RemoveContainer" containerID="99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1"
Jul 28 20:07:30.146510 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:07:30.146492    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1\": container with ID starting with 99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1 not found: ID does not exist" containerID="99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1"
Jul 28 20:07:30.146564 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.146516    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1"} err="failed to get container status \"99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1\": rpc error: code = NotFound desc = could not find container \"99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1\": container with ID starting with 99b3ca2e2c238e2886859ee5f4a9458a17194d819a5e0f47dec23b3cb0d26aa1 not found: ID does not exist"
Jul 28 20:07:30.146564 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.146533    2568 scope.go:117] "RemoveContainer" containerID="ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32"
Jul 28 20:07:30.151943 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.151917    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"]
Jul 28 20:07:30.154692 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.154673    2568 scope.go:117] "RemoveContainer" containerID="ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32"
Jul 28 20:07:30.154941 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:07:30.154922    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32\": container with ID starting with ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32 not found: ID does not exist" containerID="ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32"
Jul 28 20:07:30.154988 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.154950    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32"} err="failed to get container status \"ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32\": rpc error: code = NotFound desc = could not find container \"ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32\": container with ID starting with ceaf9c712a63dc328ceecd01aad7a876c612a99517397626d785bc3cd6343b32 not found: ID does not exist"
Jul 28 20:07:30.155546 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.155529    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg"]
Jul 28 20:07:30.165514 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.165494    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"]
Jul 28 20:07:30.172015 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.171993    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-e9667e4b-5499b8bd48-58f5q"]
Jul 28 20:07:30.173501 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.173473    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dr5vx\" (UniqueName: \"kubernetes.io/projected/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-kube-api-access-dr5vx\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:07:30.173501 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.173493    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:07:30.173501 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.173503    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/81b2217e-21c7-42ed-b787-acb9131f78b0-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:07:30.173723 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.173512    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2757f\" (UniqueName: \"kubernetes.io/projected/d466a873-99d7-46f8-ac34-59dbc47d1137-kube-api-access-2757f\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:07:30.173723 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.173520    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-62hkj\" (UniqueName: \"kubernetes.io/projected/81b2217e-21c7-42ed-b787-acb9131f78b0-kube-api-access-62hkj\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:07:30.173723 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.173529    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:07:30.173723 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.173536    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d466a873-99d7-46f8-ac34-59dbc47d1137-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:07:30.182881 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.182856    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"]
Jul 28 20:07:30.186724 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.186699    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-e9667e4b-7b7f56f966-7hgvw"]
Jul 28 20:07:30.310502 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.310426    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:30.917527 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:30.917486    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-mt-e9667e4b-74d89cb4fd-cpnkg" podUID="d466a873-99d7-46f8-ac34-59dbc47d1137" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.56:9004: i/o timeout"
Jul 28 20:07:31.676964 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:31.676930    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e512053-31ca-48a4-83bb-1700f48f8554" path="/var/lib/kubelet/pods/1e512053-31ca-48a4-83bb-1700f48f8554/volumes"
Jul 28 20:07:31.677283 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:31.677270    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81b2217e-21c7-42ed-b787-acb9131f78b0" path="/var/lib/kubelet/pods/81b2217e-21c7-42ed-b787-acb9131f78b0/volumes"
Jul 28 20:07:31.677653 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:31.677640    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" path="/var/lib/kubelet/pods/c2ef8aab-0827-4cd4-8c81-5f024a6bb06f/volumes"
Jul 28 20:07:31.677970 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:31.677958    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d466a873-99d7-46f8-ac34-59dbc47d1137" path="/var/lib/kubelet/pods/d466a873-99d7-46f8-ac34-59dbc47d1137/volumes"
Jul 28 20:07:36.296359 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:36.296306    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:36.396943 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:36.396908    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:36.587115 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:36.587027    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:36.833968 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:36.833937    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:36.968919 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:36.968885    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:37.069902 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:37.069871    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:37.218863 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:37.218826    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:42.748390 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:42.748284    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:42.809669 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:42.809630    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:42.897364 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:42.897316    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:42.992373 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:42.992340    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:43.140315 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:43.140289    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:49.060537 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:49.060499    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:49.135804 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:49.135773    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:56.231231 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:56.231196    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:56.576230 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:56.576142    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:56.882382 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:56.882269    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:59.137478 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.137442    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:59.224560 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.224525    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"]
Jul 28 20:07:59.224995 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.224976    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e512053-31ca-48a4-83bb-1700f48f8554" containerName="revoke-keys"
Jul 28 20:07:59.225089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.224998    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e512053-31ca-48a4-83bb-1700f48f8554" containerName="revoke-keys"
Jul 28 20:07:59.225089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225029    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81b2217e-21c7-42ed-b787-acb9131f78b0" containerName="payload-pre-processing"
Jul 28 20:07:59.225089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225038    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="81b2217e-21c7-42ed-b787-acb9131f78b0" containerName="payload-pre-processing"
Jul 28 20:07:59.225089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225055    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d466a873-99d7-46f8-ac34-59dbc47d1137" containerName="payload-processing"
Jul 28 20:07:59.225089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225064    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="d466a873-99d7-46f8-ac34-59dbc47d1137" containerName="payload-processing"
Jul 28 20:07:59.225089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225073    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" containerName="maas-api"
Jul 28 20:07:59.225089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225081    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" containerName="maas-api"
Jul 28 20:07:59.225474 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225175    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="c2ef8aab-0827-4cd4-8c81-5f024a6bb06f" containerName="maas-api"
Jul 28 20:07:59.225474 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225187    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e512053-31ca-48a4-83bb-1700f48f8554" containerName="revoke-keys"
Jul 28 20:07:59.225474 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225203    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="d466a873-99d7-46f8-ac34-59dbc47d1137" containerName="payload-processing"
Jul 28 20:07:59.225474 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225212    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="81b2217e-21c7-42ed-b787-acb9131f78b0" containerName="payload-pre-processing"
Jul 28 20:07:59.225474 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.225403    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:59.228563 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.228545    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.231400 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.231373    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-27915281-serving-cert\""
Jul 28 20:07:59.231592 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.231567    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:07:59.231713 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.231500    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:07:59.232467 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.232449    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:07:59.240946 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.240920    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"]
Jul 28 20:07:59.295119 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.295087    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:59.326938 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.326899    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc780741-d985-42fc-b268-542499b7abb4-openshift-service-ca\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.327120 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.326940    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-279sz\" (UniqueName: \"kubernetes.io/projected/dc780741-d985-42fc-b268-542499b7abb4-kube-api-access-279sz\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.327120 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.326991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc780741-d985-42fc-b268-542499b7abb4-maas-api-tls\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.395607 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.395525    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:59.427713 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.427684    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc780741-d985-42fc-b268-542499b7abb4-openshift-service-ca\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.427854 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.427725    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-279sz\" (UniqueName: \"kubernetes.io/projected/dc780741-d985-42fc-b268-542499b7abb4-kube-api-access-279sz\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.427854 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.427749    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc780741-d985-42fc-b268-542499b7abb4-maas-api-tls\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.428337 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.428305    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc780741-d985-42fc-b268-542499b7abb4-openshift-service-ca\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.430213 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.430190    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc780741-d985-42fc-b268-542499b7abb4-maas-api-tls\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.437512 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.437488    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-279sz\" (UniqueName: \"kubernetes.io/projected/dc780741-d985-42fc-b268-542499b7abb4-kube-api-access-279sz\") pod \"maas-api-e2e-mt-27915281-7b655d9687-lbfcr\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.542777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.542737    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:07:59.678627 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.678600    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"]
Jul 28 20:07:59.680626 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:07:59.680599    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddc780741_d985_42fc_b268_542499b7abb4.slice/crio-e2ba72d2908f928e5b3c9729561fafebdc1538988ac7ac86008e6b804852bc22 WatchSource:0}: Error finding container e2ba72d2908f928e5b3c9729561fafebdc1538988ac7ac86008e6b804852bc22: Status 404 returned error can't find the container with id e2ba72d2908f928e5b3c9729561fafebdc1538988ac7ac86008e6b804852bc22
Jul 28 20:07:59.838037 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.838003    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:59.937109 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.937026    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:07:59.953651 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.953602    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"]
Jul 28 20:07:59.957953 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.957934    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:07:59.960897 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.960876    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-27915281-dockercfg-qmwjg\""
Jul 28 20:07:59.961018 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.960893    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-27915281\""
Jul 28 20:07:59.971288 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:07:59.971258    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"]
Jul 28 20:08:00.010672 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.010635    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"]
Jul 28 20:08:00.014147 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.014127    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:00.024603 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.024578    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"]
Jul 28 20:08:00.033595 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.033562    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b05106-716c-4e5d-a22e-a61c5fe646eb-plugins-config-volume\") pod \"payload-processing-e2e-mt-27915281-59dc6645fd-9stgh\" (UID: \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\") " pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:00.033595 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.033598    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k4pkl\" (UniqueName: \"kubernetes.io/projected/f9b05106-716c-4e5d-a22e-a61c5fe646eb-kube-api-access-k4pkl\") pod \"payload-processing-e2e-mt-27915281-59dc6645fd-9stgh\" (UID: \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\") " pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:00.135071 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.135044    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c805ad2-74f1-4a8f-a388-f5c2d071c402-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j\" (UID: \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:00.135202 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.135092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b05106-716c-4e5d-a22e-a61c5fe646eb-plugins-config-volume\") pod \"payload-processing-e2e-mt-27915281-59dc6645fd-9stgh\" (UID: \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\") " pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:00.135202 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.135111    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k4pkl\" (UniqueName: \"kubernetes.io/projected/f9b05106-716c-4e5d-a22e-a61c5fe646eb-kube-api-access-k4pkl\") pod \"payload-processing-e2e-mt-27915281-59dc6645fd-9stgh\" (UID: \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\") " pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:00.135345 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.135214    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dt4z9\" (UniqueName: \"kubernetes.io/projected/0c805ad2-74f1-4a8f-a388-f5c2d071c402-kube-api-access-dt4z9\") pod \"payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j\" (UID: \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:00.135812 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.135789    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b05106-716c-4e5d-a22e-a61c5fe646eb-plugins-config-volume\") pod \"payload-processing-e2e-mt-27915281-59dc6645fd-9stgh\" (UID: \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\") " pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:00.144904 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.144884    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k4pkl\" (UniqueName: \"kubernetes.io/projected/f9b05106-716c-4e5d-a22e-a61c5fe646eb-kube-api-access-k4pkl\") pod \"payload-processing-e2e-mt-27915281-59dc6645fd-9stgh\" (UID: \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\") " pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:00.235816 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.235714    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c805ad2-74f1-4a8f-a388-f5c2d071c402-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j\" (UID: \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:00.235966 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.235837    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dt4z9\" (UniqueName: \"kubernetes.io/projected/0c805ad2-74f1-4a8f-a388-f5c2d071c402-kube-api-access-dt4z9\") pod \"payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j\" (UID: \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:00.236479 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.236460    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c805ad2-74f1-4a8f-a388-f5c2d071c402-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j\" (UID: \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:00.244470 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.244442    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dt4z9\" (UniqueName: \"kubernetes.io/projected/0c805ad2-74f1-4a8f-a388-f5c2d071c402-kube-api-access-dt4z9\") pod \"payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j\" (UID: \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:00.268227 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.268199    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:00.282597 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.282559    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr" event={"ID":"dc780741-d985-42fc-b268-542499b7abb4","Type":"ContainerStarted","Data":"6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5"}
Jul 28 20:08:00.282737 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.282628    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr" event={"ID":"dc780741-d985-42fc-b268-542499b7abb4","Type":"ContainerStarted","Data":"e2ba72d2908f928e5b3c9729561fafebdc1538988ac7ac86008e6b804852bc22"}
Jul 28 20:08:00.282804 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.282757    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:08:00.302337 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.302262    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr" podStartSLOduration=0.858040951 podStartE2EDuration="1.302242s" totalImagesPullingTime="444.201049ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:08:00.301986252 +0000 UTC m=+1993.113548136" watchObservedRunningTime="2026-07-28 20:08:00.302242 +0000 UTC m=+1993.113803865"
Jul 28 20:08:00.324447 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.324201    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:00.421069 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.421041    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"]
Jul 28 20:08:00.480044 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:00.480015    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"]
Jul 28 20:08:00.482278 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:08:00.482253    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c805ad2_74f1_4a8f_a388_f5c2d071c402.slice/crio-e90c48a2def677b770eda21b356680b4ec590208a430bc13d5e413829b402f2b WatchSource:0}: Error finding container e90c48a2def677b770eda21b356680b4ec590208a430bc13d5e413829b402f2b: Status 404 returned error can't find the container with id e90c48a2def677b770eda21b356680b4ec590208a430bc13d5e413829b402f2b
Jul 28 20:08:01.289065 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:01.289016    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j" event={"ID":"0c805ad2-74f1-4a8f-a388-f5c2d071c402","Type":"ContainerStarted","Data":"bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916"}
Jul 28 20:08:01.289573 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:01.289154    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:01.289573 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:01.289197    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j" event={"ID":"0c805ad2-74f1-4a8f-a388-f5c2d071c402","Type":"ContainerStarted","Data":"e90c48a2def677b770eda21b356680b4ec590208a430bc13d5e413829b402f2b"}
Jul 28 20:08:01.290654 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:01.290630    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh" event={"ID":"f9b05106-716c-4e5d-a22e-a61c5fe646eb","Type":"ContainerStarted","Data":"0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f"}
Jul 28 20:08:01.290778 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:01.290678    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh" event={"ID":"f9b05106-716c-4e5d-a22e-a61c5fe646eb","Type":"ContainerStarted","Data":"f46fd0dce31211a65003d2e734562dd4af76a34fd6086072358293303a6ffd20"}
Jul 28 20:08:01.290832 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:01.290791    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:01.307118 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:01.307076    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j" podStartSLOduration=2.307062848 podStartE2EDuration="2.307062848s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:08:01.30508237 +0000 UTC m=+1994.116644243" watchObservedRunningTime="2026-07-28 20:08:01.307062848 +0000 UTC m=+1994.118624710"
Jul 28 20:08:01.322066 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:01.322025    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh" podStartSLOduration=2.322012765 podStartE2EDuration="2.322012765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:08:01.321204266 +0000 UTC m=+1994.132766121" watchObservedRunningTime="2026-07-28 20:08:01.322012765 +0000 UTC m=+1994.133574628"
Jul 28 20:08:06.296593 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:06.296562    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:08:12.297821 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:12.297791    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:08:12.298254 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:12.297867    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:08:24.197462 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:24.197429    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:08:35.671608 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:08:35.671571    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:00.836772 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:00.836739    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:00.918797 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:00.918757    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:09.193032 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.192992    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"]
Jul 28 20:09:09.196624 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.196602    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"
Jul 28 20:09:09.198818 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.198795    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:09:09.202761 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.202741    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"]
Jul 28 20:09:09.254877 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.254838    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca04383-61a5-4fa3-abaa-42c7c56e116d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-27915281-phwgt\" (UID: \"4ca04383-61a5-4fa3-abaa-42c7c56e116d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"
Jul 28 20:09:09.356050 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.356007    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca04383-61a5-4fa3-abaa-42c7c56e116d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-27915281-phwgt\" (UID: \"4ca04383-61a5-4fa3-abaa-42c7c56e116d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"
Jul 28 20:09:09.356680 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.356660    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca04383-61a5-4fa3-abaa-42c7c56e116d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-27915281-phwgt\" (UID: \"4ca04383-61a5-4fa3-abaa-42c7c56e116d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"
Jul 28 20:09:09.509965 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.509873    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"
Jul 28 20:09:09.643757 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:09.643733    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"]
Jul 28 20:09:09.646172 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:09:09.646143    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ca04383_61a5_4fa3_abaa_42c7c56e116d.slice/crio-f05792f01b21499a4f59c9e870590beeaec871719ca840451274cbdf9e40640c WatchSource:0}: Error finding container f05792f01b21499a4f59c9e870590beeaec871719ca840451274cbdf9e40640c: Status 404 returned error can't find the container with id f05792f01b21499a4f59c9e870590beeaec871719ca840451274cbdf9e40640c
Jul 28 20:09:10.649686 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:10.649650    2568 generic.go:356] "Generic (PLEG): container finished" podID="4ca04383-61a5-4fa3-abaa-42c7c56e116d" containerID="55034fd414accc78e18de7fb7dce8d1f924e62caef46b202b1e83fcf730bce65" exitCode=0
Jul 28 20:09:10.650062 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:10.649699    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt" event={"ID":"4ca04383-61a5-4fa3-abaa-42c7c56e116d","Type":"ContainerDied","Data":"55034fd414accc78e18de7fb7dce8d1f924e62caef46b202b1e83fcf730bce65"}
Jul 28 20:09:10.650062 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:10.649769    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt" event={"ID":"4ca04383-61a5-4fa3-abaa-42c7c56e116d","Type":"ContainerStarted","Data":"f05792f01b21499a4f59c9e870590beeaec871719ca840451274cbdf9e40640c"}
Jul 28 20:09:11.787049 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:11.787022    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"
Jul 28 20:09:11.874163 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:11.874127    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca04383-61a5-4fa3-abaa-42c7c56e116d-maas-api-service-ca\") pod \"4ca04383-61a5-4fa3-abaa-42c7c56e116d\" (UID: \"4ca04383-61a5-4fa3-abaa-42c7c56e116d\") "
Jul 28 20:09:11.874536 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:11.874509    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ca04383-61a5-4fa3-abaa-42c7c56e116d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "4ca04383-61a5-4fa3-abaa-42c7c56e116d" (UID: "4ca04383-61a5-4fa3-abaa-42c7c56e116d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:09:11.975391 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:11.975278    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/4ca04383-61a5-4fa3-abaa-42c7c56e116d-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:09:12.660997 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:12.660965    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"
Jul 28 20:09:12.660997 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:12.660987    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt" event={"ID":"4ca04383-61a5-4fa3-abaa-42c7c56e116d","Type":"ContainerDied","Data":"f05792f01b21499a4f59c9e870590beeaec871719ca840451274cbdf9e40640c"}
Jul 28 20:09:12.661223 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:12.661039    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f05792f01b21499a4f59c9e870590beeaec871719ca840451274cbdf9e40640c"
Jul 28 20:09:24.803201 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.803160    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"]
Jul 28 20:09:24.803681 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.803412    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr" podUID="dc780741-d985-42fc-b268-542499b7abb4" containerName="maas-api" containerID="cri-o://6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5" gracePeriod=30
Jul 28 20:09:24.826100 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.826075    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:24.861950 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.861912    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"]
Jul 28 20:09:24.862170 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.862143    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh" podUID="f9b05106-716c-4e5d-a22e-a61c5fe646eb" containerName="payload-processing" containerID="cri-o://0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f" gracePeriod=30
Jul 28 20:09:24.871540 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.871399    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"]
Jul 28 20:09:24.871941 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.871910    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j" podUID="0c805ad2-74f1-4a8f-a388-f5c2d071c402" containerName="payload-pre-processing" containerID="cri-o://bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916" gracePeriod=30
Jul 28 20:09:24.901311 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.901274    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:24.998954 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:24.998920    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:25.007704 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.007633    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"]
Jul 28 20:09:25.010383 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.010355    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-27915281-phwgt"]
Jul 28 20:09:25.056392 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.056260    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:25.152507 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.152477    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:09:25.169160 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.169132    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:09:25.177072 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.177050    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:09:25.192363 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.192339    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b05106-716c-4e5d-a22e-a61c5fe646eb-plugins-config-volume\") pod \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\" (UID: \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\") "
Jul 28 20:09:25.192498 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.192378    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k4pkl\" (UniqueName: \"kubernetes.io/projected/f9b05106-716c-4e5d-a22e-a61c5fe646eb-kube-api-access-k4pkl\") pod \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\" (UID: \"f9b05106-716c-4e5d-a22e-a61c5fe646eb\") "
Jul 28 20:09:25.192746 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.192728    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b05106-716c-4e5d-a22e-a61c5fe646eb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f9b05106-716c-4e5d-a22e-a61c5fe646eb" (UID: "f9b05106-716c-4e5d-a22e-a61c5fe646eb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:09:25.194587 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.194566    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b05106-716c-4e5d-a22e-a61c5fe646eb-kube-api-access-k4pkl" (OuterVolumeSpecName: "kube-api-access-k4pkl") pod "f9b05106-716c-4e5d-a22e-a61c5fe646eb" (UID: "f9b05106-716c-4e5d-a22e-a61c5fe646eb"). InnerVolumeSpecName "kube-api-access-k4pkl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:09:25.293917 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.293881    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc780741-d985-42fc-b268-542499b7abb4-maas-api-tls\") pod \"dc780741-d985-42fc-b268-542499b7abb4\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") "
Jul 28 20:09:25.293917 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.293921    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc780741-d985-42fc-b268-542499b7abb4-openshift-service-ca\") pod \"dc780741-d985-42fc-b268-542499b7abb4\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") "
Jul 28 20:09:25.294122 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.293963    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-279sz\" (UniqueName: \"kubernetes.io/projected/dc780741-d985-42fc-b268-542499b7abb4-kube-api-access-279sz\") pod \"dc780741-d985-42fc-b268-542499b7abb4\" (UID: \"dc780741-d985-42fc-b268-542499b7abb4\") "
Jul 28 20:09:25.294122 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.293988    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dt4z9\" (UniqueName: \"kubernetes.io/projected/0c805ad2-74f1-4a8f-a388-f5c2d071c402-kube-api-access-dt4z9\") pod \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\" (UID: \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\") "
Jul 28 20:09:25.294122 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.294090    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c805ad2-74f1-4a8f-a388-f5c2d071c402-plugins-config-volume\") pod \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\" (UID: \"0c805ad2-74f1-4a8f-a388-f5c2d071c402\") "
Jul 28 20:09:25.294429 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.294404    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b05106-716c-4e5d-a22e-a61c5fe646eb-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:09:25.294518 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.294434    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k4pkl\" (UniqueName: \"kubernetes.io/projected/f9b05106-716c-4e5d-a22e-a61c5fe646eb-kube-api-access-k4pkl\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:09:25.294518 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.294446    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dc780741-d985-42fc-b268-542499b7abb4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "dc780741-d985-42fc-b268-542499b7abb4" (UID: "dc780741-d985-42fc-b268-542499b7abb4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:09:25.294623 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.294524    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c805ad2-74f1-4a8f-a388-f5c2d071c402-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0c805ad2-74f1-4a8f-a388-f5c2d071c402" (UID: "0c805ad2-74f1-4a8f-a388-f5c2d071c402"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:09:25.296161 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.296139    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c805ad2-74f1-4a8f-a388-f5c2d071c402-kube-api-access-dt4z9" (OuterVolumeSpecName: "kube-api-access-dt4z9") pod "0c805ad2-74f1-4a8f-a388-f5c2d071c402" (UID: "0c805ad2-74f1-4a8f-a388-f5c2d071c402"). InnerVolumeSpecName "kube-api-access-dt4z9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:09:25.296264 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.296157    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc780741-d985-42fc-b268-542499b7abb4-kube-api-access-279sz" (OuterVolumeSpecName: "kube-api-access-279sz") pod "dc780741-d985-42fc-b268-542499b7abb4" (UID: "dc780741-d985-42fc-b268-542499b7abb4"). InnerVolumeSpecName "kube-api-access-279sz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:09:25.296264 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.296226    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc780741-d985-42fc-b268-542499b7abb4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "dc780741-d985-42fc-b268-542499b7abb4" (UID: "dc780741-d985-42fc-b268-542499b7abb4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:09:25.395352 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.395297    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0c805ad2-74f1-4a8f-a388-f5c2d071c402-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:09:25.395352 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.395352    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/dc780741-d985-42fc-b268-542499b7abb4-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:09:25.395516 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.395362    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/dc780741-d985-42fc-b268-542499b7abb4-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:09:25.395516 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.395370    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-279sz\" (UniqueName: \"kubernetes.io/projected/dc780741-d985-42fc-b268-542499b7abb4-kube-api-access-279sz\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:09:25.395516 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.395379    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dt4z9\" (UniqueName: \"kubernetes.io/projected/0c805ad2-74f1-4a8f-a388-f5c2d071c402-kube-api-access-dt4z9\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:09:25.675770 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.675689    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ca04383-61a5-4fa3-abaa-42c7c56e116d" path="/var/lib/kubelet/pods/4ca04383-61a5-4fa3-abaa-42c7c56e116d/volumes"
Jul 28 20:09:25.730998 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.730967    2568 generic.go:356] "Generic (PLEG): container finished" podID="0c805ad2-74f1-4a8f-a388-f5c2d071c402" containerID="bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916" exitCode=0
Jul 28 20:09:25.731133 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.731016    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"
Jul 28 20:09:25.731133 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.731055    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j" event={"ID":"0c805ad2-74f1-4a8f-a388-f5c2d071c402","Type":"ContainerDied","Data":"bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916"}
Jul 28 20:09:25.731133 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.731114    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j" event={"ID":"0c805ad2-74f1-4a8f-a388-f5c2d071c402","Type":"ContainerDied","Data":"e90c48a2def677b770eda21b356680b4ec590208a430bc13d5e413829b402f2b"}
Jul 28 20:09:25.731307 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.731143    2568 scope.go:117] "RemoveContainer" containerID="bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916"
Jul 28 20:09:25.732564 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.732537    2568 generic.go:356] "Generic (PLEG): container finished" podID="f9b05106-716c-4e5d-a22e-a61c5fe646eb" containerID="0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f" exitCode=0
Jul 28 20:09:25.732661 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.732574    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh" event={"ID":"f9b05106-716c-4e5d-a22e-a61c5fe646eb","Type":"ContainerDied","Data":"0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f"}
Jul 28 20:09:25.732661 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.732636    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh" event={"ID":"f9b05106-716c-4e5d-a22e-a61c5fe646eb","Type":"ContainerDied","Data":"f46fd0dce31211a65003d2e734562dd4af76a34fd6086072358293303a6ffd20"}
Jul 28 20:09:25.732661 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.732582    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"
Jul 28 20:09:25.734153 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.734131    2568 generic.go:356] "Generic (PLEG): container finished" podID="dc780741-d985-42fc-b268-542499b7abb4" containerID="6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5" exitCode=0
Jul 28 20:09:25.734263 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.734167    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"
Jul 28 20:09:25.734263 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.734223    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr" event={"ID":"dc780741-d985-42fc-b268-542499b7abb4","Type":"ContainerDied","Data":"6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5"}
Jul 28 20:09:25.734403 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.734268    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr" event={"ID":"dc780741-d985-42fc-b268-542499b7abb4","Type":"ContainerDied","Data":"e2ba72d2908f928e5b3c9729561fafebdc1538988ac7ac86008e6b804852bc22"}
Jul 28 20:09:25.740777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.740759    2568 scope.go:117] "RemoveContainer" containerID="bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916"
Jul 28 20:09:25.741107 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:09:25.741087    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916\": container with ID starting with bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916 not found: ID does not exist" containerID="bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916"
Jul 28 20:09:25.741204 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.741112    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916"} err="failed to get container status \"bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916\": rpc error: code = NotFound desc = could not find container \"bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916\": container with ID starting with bc3773b0ff209cfa998e2383d9862b25e6d296f4f20dd20ea920aaaae7fc7916 not found: ID does not exist"
Jul 28 20:09:25.741204 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.741130    2568 scope.go:117] "RemoveContainer" containerID="0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f"
Jul 28 20:09:25.748602 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.748579    2568 scope.go:117] "RemoveContainer" containerID="0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f"
Jul 28 20:09:25.748847 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:09:25.748829    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f\": container with ID starting with 0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f not found: ID does not exist" containerID="0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f"
Jul 28 20:09:25.748903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.748855    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f"} err="failed to get container status \"0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f\": rpc error: code = NotFound desc = could not find container \"0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f\": container with ID starting with 0d81d47f05ab82a9fb0cdb2e2fd78019ae249e6fd279149b2bd0dfde89ebc63f not found: ID does not exist"
Jul 28 20:09:25.748903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.748876    2568 scope.go:117] "RemoveContainer" containerID="6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5"
Jul 28 20:09:25.755257 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.755237    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"]
Jul 28 20:09:25.756443 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.756347    2568 scope.go:117] "RemoveContainer" containerID="6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5"
Jul 28 20:09:25.756685 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:09:25.756662    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5\": container with ID starting with 6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5 not found: ID does not exist" containerID="6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5"
Jul 28 20:09:25.756778 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.756693    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5"} err="failed to get container status \"6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5\": rpc error: code = NotFound desc = could not find container \"6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5\": container with ID starting with 6d6b9148a28bdc8a1ed3cb4f45eba23a53981f56ab97384baf1c69a1f24702c5 not found: ID does not exist"
Jul 28 20:09:25.759024 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.759003    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-27915281-6b894d84d-jsx5j"]
Jul 28 20:09:25.768114 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.768094    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"]
Jul 28 20:09:25.772141 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.772116    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-27915281-7b655d9687-lbfcr"]
Jul 28 20:09:25.781891 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.781872    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"]
Jul 28 20:09:25.784820 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:25.784801    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-27915281-59dc6645fd-9stgh"]
Jul 28 20:09:27.677872 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:27.677835    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c805ad2-74f1-4a8f-a388-f5c2d071c402" path="/var/lib/kubelet/pods/0c805ad2-74f1-4a8f-a388-f5c2d071c402/volumes"
Jul 28 20:09:27.678402 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:27.678386    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dc780741-d985-42fc-b268-542499b7abb4" path="/var/lib/kubelet/pods/dc780741-d985-42fc-b268-542499b7abb4/volumes"
Jul 28 20:09:27.678737 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:27.678725    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9b05106-716c-4e5d-a22e-a61c5fe646eb" path="/var/lib/kubelet/pods/f9b05106-716c-4e5d-a22e-a61c5fe646eb/volumes"
Jul 28 20:09:28.497681 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:28.497642    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:33.501179 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:33.501139    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:09:47.704508 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:47.704477    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:09:47.705036 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:47.704937    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:09:47.803732 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:09:47.803702    2568 scope.go:117] "RemoveContainer" containerID="3d6390a8c6fb0b58b32738aa9fd601667afa3bf497dbd38a6265ff5c08532bed"
Jul 28 20:10:10.063182 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:10.063151    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:10.407623 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:10.407588    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:15.854207 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:15.854174    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:21.369511 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:21.369474    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:30.160834 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:30.160802    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:30.260704 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:30.260675    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:30.464558 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:30.464470    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:30.736147 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:30.736058    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:31.077025 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:31.076924    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:31.882678 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:31.882636    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:32.017581 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:32.017549    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:32.227849 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:32.227814    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:32.931950 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:32.931905    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:33.030873 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:33.030827    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:33.131507 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:33.131473    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:34.645845 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:34.645812    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:40.329156 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:40.329079    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:40.392569 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:40.392522    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:41.022424 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:41.022386    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:47.234703 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:47.234673    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:47.347918 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:47.347884    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:47.438264 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:47.438233    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:47.836668 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:47.836632    2568 scope.go:117] "RemoveContainer" containerID="cd5b7f2564c303e23d7214822ee8680d80adf49c3c8fd6c9f41d3fd9439a1896"
Jul 28 20:10:47.845030 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:47.845010    2568 scope.go:117] "RemoveContainer" containerID="a8814df34add1f9e6db083839832be59a10f703b87f414f7e55ea0285fe8d1ce"
Jul 28 20:10:58.010531 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:58.010491    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:10:58.384644 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:10:58.384549    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:19.471888 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:19.471842    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:25.887528 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:25.887490    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:25.990518 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:25.990484    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:26.282115 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:26.282022    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:26.625775 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:26.625687    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:26.741148 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:26.741108    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:29.184043 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.184009    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:29.264209 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.264173    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:29.264966 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.264944    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"]
Jul 28 20:11:29.265311 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265291    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9b05106-716c-4e5d-a22e-a61c5fe646eb" containerName="payload-processing"
Jul 28 20:11:29.265311 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265308    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9b05106-716c-4e5d-a22e-a61c5fe646eb" containerName="payload-processing"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265345    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dc780741-d985-42fc-b268-542499b7abb4" containerName="maas-api"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265354    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dc780741-d985-42fc-b268-542499b7abb4" containerName="maas-api"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265368    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c805ad2-74f1-4a8f-a388-f5c2d071c402" containerName="payload-pre-processing"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265373    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c805ad2-74f1-4a8f-a388-f5c2d071c402" containerName="payload-pre-processing"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265379    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ca04383-61a5-4fa3-abaa-42c7c56e116d" containerName="revoke-keys"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265384    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ca04383-61a5-4fa3-abaa-42c7c56e116d" containerName="revoke-keys"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265461    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c805ad2-74f1-4a8f-a388-f5c2d071c402" containerName="payload-pre-processing"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265472    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ca04383-61a5-4fa3-abaa-42c7c56e116d" containerName="revoke-keys"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265480    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dc780741-d985-42fc-b268-542499b7abb4" containerName="maas-api"
Jul 28 20:11:29.265533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.265492    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9b05106-716c-4e5d-a22e-a61c5fe646eb" containerName="payload-processing"
Jul 28 20:11:29.268576 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.268559    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.271140 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.271116    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:11:29.271275 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.271158    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-188d8c-serving-cert\""
Jul 28 20:11:29.271275 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.271216    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:11:29.272272 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.272250    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:11:29.278961 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.278938    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"]
Jul 28 20:11:29.300401 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.300370    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc204e2-aaf6-4278-962d-0dab1cec7f63-openshift-service-ca\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.300516 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.300495    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wfpk5\" (UniqueName: \"kubernetes.io/projected/3cc204e2-aaf6-4278-962d-0dab1cec7f63-kube-api-access-wfpk5\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.300558 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.300530    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc204e2-aaf6-4278-962d-0dab1cec7f63-maas-api-tls\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.338768 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.338739    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:29.401659 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.401627    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wfpk5\" (UniqueName: \"kubernetes.io/projected/3cc204e2-aaf6-4278-962d-0dab1cec7f63-kube-api-access-wfpk5\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.401806 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.401668    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc204e2-aaf6-4278-962d-0dab1cec7f63-maas-api-tls\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.401806 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.401711    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc204e2-aaf6-4278-962d-0dab1cec7f63-openshift-service-ca\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.402304 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.402286    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc204e2-aaf6-4278-962d-0dab1cec7f63-openshift-service-ca\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.404172 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.404150    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc204e2-aaf6-4278-962d-0dab1cec7f63-maas-api-tls\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.409757 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.409735    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wfpk5\" (UniqueName: \"kubernetes.io/projected/3cc204e2-aaf6-4278-962d-0dab1cec7f63-kube-api-access-wfpk5\") pod \"maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.435821 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.435757    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:29.580186 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.580147    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:29.727719 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.727625    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"]
Jul 28 20:11:29.730508 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:11:29.730482    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc204e2_aaf6_4278_962d_0dab1cec7f63.slice/crio-0a4742f47a72b3e680677641178e7a5e89aeb7111573322f0c717647bc225820 WatchSource:0}: Error finding container 0a4742f47a72b3e680677641178e7a5e89aeb7111573322f0c717647bc225820: Status 404 returned error can't find the container with id 0a4742f47a72b3e680677641178e7a5e89aeb7111573322f0c717647bc225820
Jul 28 20:11:29.885608 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.885570    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:29.996426 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.996314    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:29.996870 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:29.996848    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"]
Jul 28 20:11:30.001443 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.001423    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:30.003917 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.003895    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-188d8c\""
Jul 28 20:11:30.004047 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.003901    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-188d8c-dockercfg-ck9x4\""
Jul 28 20:11:30.007284 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.007264    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8f8r\" (UniqueName: \"kubernetes.io/projected/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-kube-api-access-g8f8r\") pod \"payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8\" (UID: \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:30.007401 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.007334    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8\" (UID: \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:30.012884 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.012863    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"]
Jul 28 20:11:30.027493 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.027460    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"]
Jul 28 20:11:30.030843 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.030826    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:30.043026 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.043006    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"]
Jul 28 20:11:30.108514 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.108481    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8f8r\" (UniqueName: \"kubernetes.io/projected/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-kube-api-access-g8f8r\") pod \"payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8\" (UID: \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:30.108666 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.108623    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8\" (UID: \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:30.108666 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.108658    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jc7cr\" (UniqueName: \"kubernetes.io/projected/13d01472-5385-4946-86d9-551d8be4f032-kube-api-access-jc7cr\") pod \"payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9\" (UID: \"13d01472-5385-4946-86d9-551d8be4f032\") " pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:30.108792 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.108776    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d01472-5385-4946-86d9-551d8be4f032-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9\" (UID: \"13d01472-5385-4946-86d9-551d8be4f032\") " pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:30.109301 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.109283    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8\" (UID: \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:30.117249 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.117221    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8f8r\" (UniqueName: \"kubernetes.io/projected/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-kube-api-access-g8f8r\") pod \"payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8\" (UID: \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:30.209920 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.209895    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jc7cr\" (UniqueName: \"kubernetes.io/projected/13d01472-5385-4946-86d9-551d8be4f032-kube-api-access-jc7cr\") pod \"payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9\" (UID: \"13d01472-5385-4946-86d9-551d8be4f032\") " pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:30.210262 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.209953    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d01472-5385-4946-86d9-551d8be4f032-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9\" (UID: \"13d01472-5385-4946-86d9-551d8be4f032\") " pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:30.210629 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.210610    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d01472-5385-4946-86d9-551d8be4f032-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9\" (UID: \"13d01472-5385-4946-86d9-551d8be4f032\") " pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:30.219038 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.219015    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jc7cr\" (UniqueName: \"kubernetes.io/projected/13d01472-5385-4946-86d9-551d8be4f032-kube-api-access-jc7cr\") pod \"payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9\" (UID: \"13d01472-5385-4946-86d9-551d8be4f032\") " pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:30.313485 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.313401    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:30.340798 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.340773    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:30.343664 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.343634    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9" event={"ID":"3cc204e2-aaf6-4278-962d-0dab1cec7f63","Type":"ContainerStarted","Data":"e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27"}
Jul 28 20:11:30.343819 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.343710    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:30.343819 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.343735    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9" event={"ID":"3cc204e2-aaf6-4278-962d-0dab1cec7f63","Type":"ContainerStarted","Data":"0a4742f47a72b3e680677641178e7a5e89aeb7111573322f0c717647bc225820"}
Jul 28 20:11:30.362365 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.362146    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9" podStartSLOduration=0.905279804 podStartE2EDuration="1.362129185s" totalImagesPullingTime="456.849381ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:11:30.359786506 +0000 UTC m=+2203.171348406" watchObservedRunningTime="2026-07-28 20:11:30.362129185 +0000 UTC m=+2203.173691051"
Jul 28 20:11:30.462574 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.462543    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"]
Jul 28 20:11:30.465178 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:11:30.465148    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ea8e0bc_f36e_43a0_bcdb_2ba1aa659610.slice/crio-bb708e809bed924e865905ff7b1d1b773b503b6396be9c93d100649f3e2c6536 WatchSource:0}: Error finding container bb708e809bed924e865905ff7b1d1b773b503b6396be9c93d100649f3e2c6536: Status 404 returned error can't find the container with id bb708e809bed924e865905ff7b1d1b773b503b6396be9c93d100649f3e2c6536
Jul 28 20:11:30.493836 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:30.493803    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"]
Jul 28 20:11:30.495732 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:11:30.495704    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13d01472_5385_4946_86d9_551d8be4f032.slice/crio-8218e3d4a67060cb7691adab49309b305c156b4184e2e0d59c8d15119ee7eac8 WatchSource:0}: Error finding container 8218e3d4a67060cb7691adab49309b305c156b4184e2e0d59c8d15119ee7eac8: Status 404 returned error can't find the container with id 8218e3d4a67060cb7691adab49309b305c156b4184e2e0d59c8d15119ee7eac8
Jul 28 20:11:31.350467 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:31.350428    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8" event={"ID":"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610","Type":"ContainerStarted","Data":"e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072"}
Jul 28 20:11:31.350930 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:31.350492    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8" event={"ID":"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610","Type":"ContainerStarted","Data":"bb708e809bed924e865905ff7b1d1b773b503b6396be9c93d100649f3e2c6536"}
Jul 28 20:11:31.350930 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:31.350543    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:31.351914 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:31.351884    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9" event={"ID":"13d01472-5385-4946-86d9-551d8be4f032","Type":"ContainerStarted","Data":"265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373"}
Jul 28 20:11:31.352054 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:31.351984    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:31.352054 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:31.352019    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9" event={"ID":"13d01472-5385-4946-86d9-551d8be4f032","Type":"ContainerStarted","Data":"8218e3d4a67060cb7691adab49309b305c156b4184e2e0d59c8d15119ee7eac8"}
Jul 28 20:11:31.366741 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:31.366702    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8" podStartSLOduration=2.366689852 podStartE2EDuration="2.366689852s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:11:31.366151977 +0000 UTC m=+2204.177713869" watchObservedRunningTime="2026-07-28 20:11:31.366689852 +0000 UTC m=+2204.178251714"
Jul 28 20:11:31.383190 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:31.383146    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9" podStartSLOduration=1.3831315 podStartE2EDuration="1.3831315s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:11:31.381456166 +0000 UTC m=+2204.193018030" watchObservedRunningTime="2026-07-28 20:11:31.3831315 +0000 UTC m=+2204.194693364"
Jul 28 20:11:36.357279 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:36.357247    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:11:42.358340 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:42.358286    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:11:42.358753 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:42.358465    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:11:47.865839 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:47.865807    2568 scope.go:117] "RemoveContainer" containerID="673bc8b3da3c6442318248961561499a429e65ba6bb3be36e264888cf3ca68d0"
Jul 28 20:11:54.232944 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:54.232909    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:54.597834 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:54.597750    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:54.921554 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:54.921498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:55.723084 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:55.723047    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:58.406253 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.406220    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:58.511192 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.511159    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:58.646952 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.646916    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"]
Jul 28 20:11:58.651153 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.651130    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.654892 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.654867    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ba3d8d-serving-cert\""
Jul 28 20:11:58.664704 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.664679    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"]
Jul 28 20:11:58.698403 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.698372    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:58.768934 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.768891    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.768934 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.768929    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bwk8m\" (UniqueName: \"kubernetes.io/projected/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-kube-api-access-bwk8m\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.769155 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.769040    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.795903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.795869    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:58.870129 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.870092    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.870129 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.870132    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bwk8m\" (UniqueName: \"kubernetes.io/projected/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-kube-api-access-bwk8m\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.870373 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.870180    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.870942 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.870918    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.872564 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.872539    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-maas-api-tls\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.879656 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.879630    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bwk8m\" (UniqueName: \"kubernetes.io/projected/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-kube-api-access-bwk8m\") pod \"maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:58.964971 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:58.964883    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:11:59.103765 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.103740    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"]
Jul 28 20:11:59.106086 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:11:59.106055    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddaab92d3_acf8_4f8e_9d5c_095e5fcf0082.slice/crio-bbccaacf1b83011e24f67da4e8cf7cc9510d2afdfeb800c967ce2929bb85d797 WatchSource:0}: Error finding container bbccaacf1b83011e24f67da4e8cf7cc9510d2afdfeb800c967ce2929bb85d797: Status 404 returned error can't find the container with id bbccaacf1b83011e24f67da4e8cf7cc9510d2afdfeb800c967ce2929bb85d797
Jul 28 20:11:59.107903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.107884    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:11:59.216146 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.216060    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:59.316615 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.316560    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:11:59.343249 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.343216    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"]
Jul 28 20:11:59.348021 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.348005    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:11:59.350648 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.350621    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-ba3d8d-dockercfg-bfp9z\""
Jul 28 20:11:59.350776 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.350752    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-ba3d8d\""
Jul 28 20:11:59.357621 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.357592    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"]
Jul 28 20:11:59.476608 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.476522    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x\" (UID: \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:11:59.476608 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.476561    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8jjn\" (UniqueName: \"kubernetes.io/projected/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-kube-api-access-f8jjn\") pod \"payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x\" (UID: \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:11:59.490798 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.490754    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr" event={"ID":"daab92d3-acf8-4f8e-9d5c-095e5fcf0082","Type":"ContainerStarted","Data":"bbccaacf1b83011e24f67da4e8cf7cc9510d2afdfeb800c967ce2929bb85d797"}
Jul 28 20:11:59.578183 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.578144    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x\" (UID: \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:11:59.578183 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.578181    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8jjn\" (UniqueName: \"kubernetes.io/projected/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-kube-api-access-f8jjn\") pod \"payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x\" (UID: \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:11:59.578951 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.578924    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x\" (UID: \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:11:59.593278 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.593247    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8jjn\" (UniqueName: \"kubernetes.io/projected/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-kube-api-access-f8jjn\") pod \"payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x\" (UID: \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\") " pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:11:59.659181 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.659146    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:11:59.806064 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:11:59.806039    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"]
Jul 28 20:11:59.808251 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:11:59.808222    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6ffe1a7_6c8c_437a_bfac_3b6bf07beb90.slice/crio-7ef8935482bcb746d6c4fcdf2b3d0dcd5de022d4021d314f00351735f7d38895 WatchSource:0}: Error finding container 7ef8935482bcb746d6c4fcdf2b3d0dcd5de022d4021d314f00351735f7d38895: Status 404 returned error can't find the container with id 7ef8935482bcb746d6c4fcdf2b3d0dcd5de022d4021d314f00351735f7d38895
Jul 28 20:12:00.498491 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:00.498455    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x" event={"ID":"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90","Type":"ContainerStarted","Data":"92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe"}
Jul 28 20:12:00.499050 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:00.498509    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x" event={"ID":"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90","Type":"ContainerStarted","Data":"7ef8935482bcb746d6c4fcdf2b3d0dcd5de022d4021d314f00351735f7d38895"}
Jul 28 20:12:00.499050 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:00.498569    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:12:00.500053 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:00.500030    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr" event={"ID":"daab92d3-acf8-4f8e-9d5c-095e5fcf0082","Type":"ContainerStarted","Data":"b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c"}
Jul 28 20:12:00.500178 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:00.500160    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:12:00.534059 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:00.533983    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x" podStartSLOduration=1.5339687899999999 podStartE2EDuration="1.53396879s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:12:00.532886798 +0000 UTC m=+2233.344448673" watchObservedRunningTime="2026-07-28 20:12:00.53396879 +0000 UTC m=+2233.345530653"
Jul 28 20:12:00.559136 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:00.559081    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr" podStartSLOduration=1.394912006 podStartE2EDuration="2.559069136s" totalImagesPullingTime="1.16415713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:11:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:12:00.558794532 +0000 UTC m=+2233.370356423" watchObservedRunningTime="2026-07-28 20:12:00.559069136 +0000 UTC m=+2233.370631003"
Jul 28 20:12:06.511129 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:06.511097    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:12:11.507562 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:11.507478    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:12:23.471578 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:23.471522    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:12:54.654695 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:12:54.654637    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:03.497080 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:13:03.497039    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:03.662603 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:13:03.662565    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:20.893394 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:13:20.893307    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:23.791241 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:13:23.791197    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:33.978195 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:13:33.978156    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:13:47.893093 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:13:47.893020    2568 scope.go:117] "RemoveContainer" containerID="70b9387444a782e0188cb5e50cee6ef2463f25a168522ef52cb5b5ed83c99aa3"
Jul 28 20:14:04.077712 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:04.077642    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:04.997539 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:04.997510    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:05.144895 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.144863    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"]
Jul 28 20:14:05.148370 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.148352    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"
Jul 28 20:14:05.150924 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.150897    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:14:05.155917 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.155549    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"]
Jul 28 20:14:05.203028 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.202991    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a31f47-5960-4abb-a2ef-100167cd90cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr\" (UID: \"a6a31f47-5960-4abb-a2ef-100167cd90cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"
Jul 28 20:14:05.303635 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.303553    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a31f47-5960-4abb-a2ef-100167cd90cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr\" (UID: \"a6a31f47-5960-4abb-a2ef-100167cd90cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"
Jul 28 20:14:05.304167 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.304144    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a31f47-5960-4abb-a2ef-100167cd90cd-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr\" (UID: \"a6a31f47-5960-4abb-a2ef-100167cd90cd\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"
Jul 28 20:14:05.465582 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.465541    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"
Jul 28 20:14:05.598109 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:05.598083    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"]
Jul 28 20:14:05.600471 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:14:05.600443    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6a31f47_5960_4abb_a2ef_100167cd90cd.slice/crio-b81fe6047ce79b355924584feb57e5e7030dfb1ad31910f78e47cbb00343ba57 WatchSource:0}: Error finding container b81fe6047ce79b355924584feb57e5e7030dfb1ad31910f78e47cbb00343ba57: Status 404 returned error can't find the container with id b81fe6047ce79b355924584feb57e5e7030dfb1ad31910f78e47cbb00343ba57
Jul 28 20:14:06.116976 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:06.116933    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr" event={"ID":"a6a31f47-5960-4abb-a2ef-100167cd90cd","Type":"ContainerStarted","Data":"4d5ae7546a104e2c6a1a07bc75428bb43740a88085f14aaba3179973692e5e48"}
Jul 28 20:14:06.117179 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:06.116990    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr" event={"ID":"a6a31f47-5960-4abb-a2ef-100167cd90cd","Type":"ContainerStarted","Data":"b81fe6047ce79b355924584feb57e5e7030dfb1ad31910f78e47cbb00343ba57"}
Jul 28 20:14:06.131905 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:06.131854    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr" podStartSLOduration=1.131838658 podStartE2EDuration="1.131838658s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:14:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:14:06.130944302 +0000 UTC m=+2358.942506166" watchObservedRunningTime="2026-07-28 20:14:06.131838658 +0000 UTC m=+2358.943400520"
Jul 28 20:14:07.124460 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:07.124425    2568 generic.go:356] "Generic (PLEG): container finished" podID="a6a31f47-5960-4abb-a2ef-100167cd90cd" containerID="4d5ae7546a104e2c6a1a07bc75428bb43740a88085f14aaba3179973692e5e48" exitCode=0
Jul 28 20:14:07.124900 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:07.124510    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr" event={"ID":"a6a31f47-5960-4abb-a2ef-100167cd90cd","Type":"ContainerDied","Data":"4d5ae7546a104e2c6a1a07bc75428bb43740a88085f14aaba3179973692e5e48"}
Jul 28 20:14:08.264733 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:08.264705    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"
Jul 28 20:14:08.331091 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:08.331058    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a31f47-5960-4abb-a2ef-100167cd90cd-maas-api-service-ca\") pod \"a6a31f47-5960-4abb-a2ef-100167cd90cd\" (UID: \"a6a31f47-5960-4abb-a2ef-100167cd90cd\") "
Jul 28 20:14:08.331417 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:08.331394    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6a31f47-5960-4abb-a2ef-100167cd90cd-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a6a31f47-5960-4abb-a2ef-100167cd90cd" (UID: "a6a31f47-5960-4abb-a2ef-100167cd90cd"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:08.432536 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:08.432492    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6a31f47-5960-4abb-a2ef-100167cd90cd-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:09.135497 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:09.135466    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"
Jul 28 20:14:09.135497 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:09.135484    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr" event={"ID":"a6a31f47-5960-4abb-a2ef-100167cd90cd","Type":"ContainerDied","Data":"b81fe6047ce79b355924584feb57e5e7030dfb1ad31910f78e47cbb00343ba57"}
Jul 28 20:14:09.135755 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:09.135526    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b81fe6047ce79b355924584feb57e5e7030dfb1ad31910f78e47cbb00343ba57"
Jul 28 20:14:15.698924 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:15.698883    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"]
Jul 28 20:14:15.699456 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:15.699086    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9" podUID="3cc204e2-aaf6-4278-962d-0dab1cec7f63" containerName="maas-api" containerID="cri-o://e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27" gracePeriod=30
Jul 28 20:14:15.760151 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:15.760111    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"]
Jul 28 20:14:15.760412 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:15.760381    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9" podUID="13d01472-5385-4946-86d9-551d8be4f032" containerName="payload-processing" containerID="cri-o://265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373" gracePeriod=30
Jul 28 20:14:15.772036 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:15.772000    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"]
Jul 28 20:14:15.772450 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:15.772397    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8" podUID="4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610" containerName="payload-pre-processing" containerID="cri-o://e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072" gracePeriod=30
Jul 28 20:14:15.923041 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:15.922975    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"]
Jul 28 20:14:15.925049 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:15.925008    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-188d8c-jvrmr"]
Jul 28 20:14:16.065185 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.065082    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:14:16.068646 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.068607    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:14:16.145809 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.145784    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:14:16.172660 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.172628    2568 generic.go:356] "Generic (PLEG): container finished" podID="4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610" containerID="e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072" exitCode=0
Jul 28 20:14:16.172849 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.172686    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"
Jul 28 20:14:16.172849 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.172740    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8" event={"ID":"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610","Type":"ContainerDied","Data":"e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072"}
Jul 28 20:14:16.172849 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.172820    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8" event={"ID":"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610","Type":"ContainerDied","Data":"bb708e809bed924e865905ff7b1d1b773b503b6396be9c93d100649f3e2c6536"}
Jul 28 20:14:16.173025 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.172860    2568 scope.go:117] "RemoveContainer" containerID="e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072"
Jul 28 20:14:16.174345 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.174304    2568 generic.go:356] "Generic (PLEG): container finished" podID="13d01472-5385-4946-86d9-551d8be4f032" containerID="265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373" exitCode=0
Jul 28 20:14:16.174483 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.174360    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9" event={"ID":"13d01472-5385-4946-86d9-551d8be4f032","Type":"ContainerDied","Data":"265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373"}
Jul 28 20:14:16.174483 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.174411    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9" event={"ID":"13d01472-5385-4946-86d9-551d8be4f032","Type":"ContainerDied","Data":"8218e3d4a67060cb7691adab49309b305c156b4184e2e0d59c8d15119ee7eac8"}
Jul 28 20:14:16.174483 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.174369    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"
Jul 28 20:14:16.176086 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.176063    2568 generic.go:356] "Generic (PLEG): container finished" podID="3cc204e2-aaf6-4278-962d-0dab1cec7f63" containerID="e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27" exitCode=0
Jul 28 20:14:16.176216 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.176107    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9" event={"ID":"3cc204e2-aaf6-4278-962d-0dab1cec7f63","Type":"ContainerDied","Data":"e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27"}
Jul 28 20:14:16.176216 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.176112    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"
Jul 28 20:14:16.176216 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.176143    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9" event={"ID":"3cc204e2-aaf6-4278-962d-0dab1cec7f63","Type":"ContainerDied","Data":"0a4742f47a72b3e680677641178e7a5e89aeb7111573322f0c717647bc225820"}
Jul 28 20:14:16.183924 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.183906    2568 scope.go:117] "RemoveContainer" containerID="e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072"
Jul 28 20:14:16.184177 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:14:16.184145    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072\": container with ID starting with e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072 not found: ID does not exist" containerID="e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072"
Jul 28 20:14:16.184261 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.184180    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072"} err="failed to get container status \"e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072\": rpc error: code = NotFound desc = could not find container \"e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072\": container with ID starting with e2afdf180abb5f26c048b2bf6624f4ed8014eb2e33dffd19577155098acfa072 not found: ID does not exist"
Jul 28 20:14:16.184261 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.184203    2568 scope.go:117] "RemoveContainer" containerID="265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373"
Jul 28 20:14:16.192072 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.192057    2568 scope.go:117] "RemoveContainer" containerID="265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373"
Jul 28 20:14:16.192295 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:14:16.192277    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373\": container with ID starting with 265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373 not found: ID does not exist" containerID="265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373"
Jul 28 20:14:16.192368 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.192304    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373"} err="failed to get container status \"265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373\": rpc error: code = NotFound desc = could not find container \"265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373\": container with ID starting with 265e5d8ef2c1019dfca8c1b9a7137a86d75229bec003c353be570bf97389e373 not found: ID does not exist"
Jul 28 20:14:16.192368 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.192339    2568 scope.go:117] "RemoveContainer" containerID="e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27"
Jul 28 20:14:16.200208 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.200189    2568 scope.go:117] "RemoveContainer" containerID="e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27"
Jul 28 20:14:16.200471 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:14:16.200453    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27\": container with ID starting with e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27 not found: ID does not exist" containerID="e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27"
Jul 28 20:14:16.200551 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.200477    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27"} err="failed to get container status \"e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27\": rpc error: code = NotFound desc = could not find container \"e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27\": container with ID starting with e702a4564fd5c712e7a53f1d655ad3fda62e39c6ba30bf37b337f5e263c17d27 not found: ID does not exist"
Jul 28 20:14:16.201774 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.201752    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d01472-5385-4946-86d9-551d8be4f032-plugins-config-volume\") pod \"13d01472-5385-4946-86d9-551d8be4f032\" (UID: \"13d01472-5385-4946-86d9-551d8be4f032\") "
Jul 28 20:14:16.201881 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.201847    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wfpk5\" (UniqueName: \"kubernetes.io/projected/3cc204e2-aaf6-4278-962d-0dab1cec7f63-kube-api-access-wfpk5\") pod \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") "
Jul 28 20:14:16.201881 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.201865    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc204e2-aaf6-4278-962d-0dab1cec7f63-maas-api-tls\") pod \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") "
Jul 28 20:14:16.201993 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.201889    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g8f8r\" (UniqueName: \"kubernetes.io/projected/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-kube-api-access-g8f8r\") pod \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\" (UID: \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\") "
Jul 28 20:14:16.201993 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.201930    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jc7cr\" (UniqueName: \"kubernetes.io/projected/13d01472-5385-4946-86d9-551d8be4f032-kube-api-access-jc7cr\") pod \"13d01472-5385-4946-86d9-551d8be4f032\" (UID: \"13d01472-5385-4946-86d9-551d8be4f032\") "
Jul 28 20:14:16.202110 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.202038    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc204e2-aaf6-4278-962d-0dab1cec7f63-openshift-service-ca\") pod \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\" (UID: \"3cc204e2-aaf6-4278-962d-0dab1cec7f63\") "
Jul 28 20:14:16.202110 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.202076    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-plugins-config-volume\") pod \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\" (UID: \"4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610\") "
Jul 28 20:14:16.202213 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.202152    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13d01472-5385-4946-86d9-551d8be4f032-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13d01472-5385-4946-86d9-551d8be4f032" (UID: "13d01472-5385-4946-86d9-551d8be4f032"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:16.202431 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.202391    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3cc204e2-aaf6-4278-962d-0dab1cec7f63-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3cc204e2-aaf6-4278-962d-0dab1cec7f63" (UID: "3cc204e2-aaf6-4278-962d-0dab1cec7f63"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:16.202524 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.202466    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13d01472-5385-4946-86d9-551d8be4f032-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:16.202641 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.202622    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610" (UID: "4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:16.204016 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.203994    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13d01472-5385-4946-86d9-551d8be4f032-kube-api-access-jc7cr" (OuterVolumeSpecName: "kube-api-access-jc7cr") pod "13d01472-5385-4946-86d9-551d8be4f032" (UID: "13d01472-5385-4946-86d9-551d8be4f032"). InnerVolumeSpecName "kube-api-access-jc7cr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:14:16.204156 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.204142    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3cc204e2-aaf6-4278-962d-0dab1cec7f63-kube-api-access-wfpk5" (OuterVolumeSpecName: "kube-api-access-wfpk5") pod "3cc204e2-aaf6-4278-962d-0dab1cec7f63" (UID: "3cc204e2-aaf6-4278-962d-0dab1cec7f63"). InnerVolumeSpecName "kube-api-access-wfpk5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:14:16.204445 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.204423    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-kube-api-access-g8f8r" (OuterVolumeSpecName: "kube-api-access-g8f8r") pod "4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610" (UID: "4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610"). InnerVolumeSpecName "kube-api-access-g8f8r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:14:16.204502 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.204431    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3cc204e2-aaf6-4278-962d-0dab1cec7f63-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3cc204e2-aaf6-4278-962d-0dab1cec7f63" (UID: "3cc204e2-aaf6-4278-962d-0dab1cec7f63"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:14:16.303769 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.303691    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:16.303769 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.303723    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wfpk5\" (UniqueName: \"kubernetes.io/projected/3cc204e2-aaf6-4278-962d-0dab1cec7f63-kube-api-access-wfpk5\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:16.303769 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.303734    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3cc204e2-aaf6-4278-962d-0dab1cec7f63-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:16.303769 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.303744    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g8f8r\" (UniqueName: \"kubernetes.io/projected/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610-kube-api-access-g8f8r\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:16.303769 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.303754    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jc7cr\" (UniqueName: \"kubernetes.io/projected/13d01472-5385-4946-86d9-551d8be4f032-kube-api-access-jc7cr\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:16.303769 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.303764    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3cc204e2-aaf6-4278-962d-0dab1cec7f63-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:16.418512 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.418474    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:16.475064 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.475030    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:16.498955 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.498928    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"]
Jul 28 20:14:16.502469 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.502436    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-188d8c-695bbd7db4-9vkc8"]
Jul 28 20:14:16.513578 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.513556    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"]
Jul 28 20:14:16.519361 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.519339    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-188d8c-77fdd55c8-zqms9"]
Jul 28 20:14:16.530550 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.530528    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"]
Jul 28 20:14:16.532165 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:16.532146    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-188d8c-54c7b5bf7d-6t6z9"]
Jul 28 20:14:17.676393 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:17.676358    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13d01472-5385-4946-86d9-551d8be4f032" path="/var/lib/kubelet/pods/13d01472-5385-4946-86d9-551d8be4f032/volumes"
Jul 28 20:14:17.676785 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:17.676751    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3cc204e2-aaf6-4278-962d-0dab1cec7f63" path="/var/lib/kubelet/pods/3cc204e2-aaf6-4278-962d-0dab1cec7f63/volumes"
Jul 28 20:14:17.677062 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:17.677048    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610" path="/var/lib/kubelet/pods/4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610/volumes"
Jul 28 20:14:17.677352 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:17.677317    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6a31f47-5960-4abb-a2ef-100167cd90cd" path="/var/lib/kubelet/pods/a6a31f47-5960-4abb-a2ef-100167cd90cd/volumes"
Jul 28 20:14:22.113247 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.113214    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:22.212458 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.212415    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:22.391479 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.391453    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:22.632282 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.632247    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:22.756239 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756159    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"]
Jul 28 20:14:22.756544 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756530    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3cc204e2-aaf6-4278-962d-0dab1cec7f63" containerName="maas-api"
Jul 28 20:14:22.756588 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756545    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3cc204e2-aaf6-4278-962d-0dab1cec7f63" containerName="maas-api"
Jul 28 20:14:22.756588 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756553    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610" containerName="payload-pre-processing"
Jul 28 20:14:22.756588 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756561    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610" containerName="payload-pre-processing"
Jul 28 20:14:22.756588 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756570    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13d01472-5385-4946-86d9-551d8be4f032" containerName="payload-processing"
Jul 28 20:14:22.756588 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756576    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="13d01472-5385-4946-86d9-551d8be4f032" containerName="payload-processing"
Jul 28 20:14:22.756588 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756586    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6a31f47-5960-4abb-a2ef-100167cd90cd" containerName="revoke-keys"
Jul 28 20:14:22.756790 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756593    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6a31f47-5960-4abb-a2ef-100167cd90cd" containerName="revoke-keys"
Jul 28 20:14:22.756790 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756657    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="13d01472-5385-4946-86d9-551d8be4f032" containerName="payload-processing"
Jul 28 20:14:22.756790 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756664    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6a31f47-5960-4abb-a2ef-100167cd90cd" containerName="revoke-keys"
Jul 28 20:14:22.756790 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756673    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ea8e0bc-f36e-43a0-bcdb-2ba1aa659610" containerName="payload-pre-processing"
Jul 28 20:14:22.756790 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.756682    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3cc204e2-aaf6-4278-962d-0dab1cec7f63" containerName="maas-api"
Jul 28 20:14:22.761078 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.761057    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"
Jul 28 20:14:22.763697 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.763658    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:14:22.765758 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.765727    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"]
Jul 28 20:14:22.862349 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.862293    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2476b7c-63f3-466f-8c08-065d96b3b3f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68\" (UID: \"a2476b7c-63f3-466f-8c08-065d96b3b3f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"
Jul 28 20:14:22.962971 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.962934    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2476b7c-63f3-466f-8c08-065d96b3b3f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68\" (UID: \"a2476b7c-63f3-466f-8c08-065d96b3b3f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"
Jul 28 20:14:22.963565 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:22.963547    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2476b7c-63f3-466f-8c08-065d96b3b3f6-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68\" (UID: \"a2476b7c-63f3-466f-8c08-065d96b3b3f6\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"
Jul 28 20:14:23.099637 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:23.099555    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"
Jul 28 20:14:23.231854 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:23.231824    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"]
Jul 28 20:14:23.233385 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:14:23.233306    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2476b7c_63f3_466f_8c08_065d96b3b3f6.slice/crio-a9bf085752bbe54f6c2293a620a890341813702ff49738a056c0c16b9b5ce64a WatchSource:0}: Error finding container a9bf085752bbe54f6c2293a620a890341813702ff49738a056c0c16b9b5ce64a: Status 404 returned error can't find the container with id a9bf085752bbe54f6c2293a620a890341813702ff49738a056c0c16b9b5ce64a
Jul 28 20:14:24.225548 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:24.225456    2568 generic.go:356] "Generic (PLEG): container finished" podID="a2476b7c-63f3-466f-8c08-065d96b3b3f6" containerID="5313fcf4947c31e121897c069c1d22380badfff1a69dd060d66381d0c24b1413" exitCode=0
Jul 28 20:14:24.225548 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:24.225517    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68" event={"ID":"a2476b7c-63f3-466f-8c08-065d96b3b3f6","Type":"ContainerDied","Data":"5313fcf4947c31e121897c069c1d22380badfff1a69dd060d66381d0c24b1413"}
Jul 28 20:14:24.225740 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:24.225566    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68" event={"ID":"a2476b7c-63f3-466f-8c08-065d96b3b3f6","Type":"ContainerStarted","Data":"a9bf085752bbe54f6c2293a620a890341813702ff49738a056c0c16b9b5ce64a"}
Jul 28 20:14:25.378591 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:25.378559    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"
Jul 28 20:14:25.483268 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:25.483234    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2476b7c-63f3-466f-8c08-065d96b3b3f6-maas-api-service-ca\") pod \"a2476b7c-63f3-466f-8c08-065d96b3b3f6\" (UID: \"a2476b7c-63f3-466f-8c08-065d96b3b3f6\") "
Jul 28 20:14:25.483622 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:25.483596    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a2476b7c-63f3-466f-8c08-065d96b3b3f6-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "a2476b7c-63f3-466f-8c08-065d96b3b3f6" (UID: "a2476b7c-63f3-466f-8c08-065d96b3b3f6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:25.584845 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:25.584755    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/a2476b7c-63f3-466f-8c08-065d96b3b3f6-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:26.240466 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:26.240431    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"
Jul 28 20:14:26.240466 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:26.240454    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68" event={"ID":"a2476b7c-63f3-466f-8c08-065d96b3b3f6","Type":"ContainerDied","Data":"a9bf085752bbe54f6c2293a620a890341813702ff49738a056c0c16b9b5ce64a"}
Jul 28 20:14:26.240675 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:26.240502    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9bf085752bbe54f6c2293a620a890341813702ff49738a056c0c16b9b5ce64a"
Jul 28 20:14:33.297250 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.297211    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"]
Jul 28 20:14:33.297705 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.297489    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr" podUID="daab92d3-acf8-4f8e-9d5c-095e5fcf0082" containerName="maas-api" containerID="cri-o://b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c" gracePeriod=30
Jul 28 20:14:33.329669 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.329636    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:33.350427 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.350398    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"]
Jul 28 20:14:33.350622 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.350598    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x" podUID="f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90" containerName="payload-processing" containerID="cri-o://92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe" gracePeriod=30
Jul 28 20:14:33.396889 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.396859    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:33.497632 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.497599    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:33.502948 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.502918    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"]
Jul 28 20:14:33.505946 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.505917    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-ba3d8d-6ns68"]
Jul 28 20:14:33.555751 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.555718    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:33.619907 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.619882    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:14:33.622849 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.622828    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:14:33.676392 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.676354    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2476b7c-63f3-466f-8c08-065d96b3b3f6" path="/var/lib/kubelet/pods/a2476b7c-63f3-466f-8c08-065d96b3b3f6/volumes"
Jul 28 20:14:33.757097 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.757064    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-maas-api-tls\") pod \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") "
Jul 28 20:14:33.757097 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.757099    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8jjn\" (UniqueName: \"kubernetes.io/projected/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-kube-api-access-f8jjn\") pod \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\" (UID: \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\") "
Jul 28 20:14:33.757303 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.757183    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-plugins-config-volume\") pod \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\" (UID: \"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90\") "
Jul 28 20:14:33.757303 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.757213    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-openshift-service-ca\") pod \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") "
Jul 28 20:14:33.757303 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.757277    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bwk8m\" (UniqueName: \"kubernetes.io/projected/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-kube-api-access-bwk8m\") pod \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\" (UID: \"daab92d3-acf8-4f8e-9d5c-095e5fcf0082\") "
Jul 28 20:14:33.757614 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.757584    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90" (UID: "f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:33.757732 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.757619    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "daab92d3-acf8-4f8e-9d5c-095e5fcf0082" (UID: "daab92d3-acf8-4f8e-9d5c-095e5fcf0082"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:14:33.759362 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.759310    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-kube-api-access-f8jjn" (OuterVolumeSpecName: "kube-api-access-f8jjn") pod "f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90" (UID: "f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90"). InnerVolumeSpecName "kube-api-access-f8jjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:14:33.759513 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.759337    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-kube-api-access-bwk8m" (OuterVolumeSpecName: "kube-api-access-bwk8m") pod "daab92d3-acf8-4f8e-9d5c-095e5fcf0082" (UID: "daab92d3-acf8-4f8e-9d5c-095e5fcf0082"). InnerVolumeSpecName "kube-api-access-bwk8m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:14:33.759513 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.759442    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "daab92d3-acf8-4f8e-9d5c-095e5fcf0082" (UID: "daab92d3-acf8-4f8e-9d5c-095e5fcf0082"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:14:33.858249 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.858152    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:33.858249 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.858187    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:33.858249 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.858201    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bwk8m\" (UniqueName: \"kubernetes.io/projected/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-kube-api-access-bwk8m\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:33.858249 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.858212    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/daab92d3-acf8-4f8e-9d5c-095e5fcf0082-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:33.858249 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.858225    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8jjn\" (UniqueName: \"kubernetes.io/projected/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90-kube-api-access-f8jjn\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:14:33.943976 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:33.943927    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:34.288246 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.288208    2568 generic.go:356] "Generic (PLEG): container finished" podID="daab92d3-acf8-4f8e-9d5c-095e5fcf0082" containerID="b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c" exitCode=0
Jul 28 20:14:34.288446 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.288256    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"
Jul 28 20:14:34.288446 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.288298    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr" event={"ID":"daab92d3-acf8-4f8e-9d5c-095e5fcf0082","Type":"ContainerDied","Data":"b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c"}
Jul 28 20:14:34.288446 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.288375    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr" event={"ID":"daab92d3-acf8-4f8e-9d5c-095e5fcf0082","Type":"ContainerDied","Data":"bbccaacf1b83011e24f67da4e8cf7cc9510d2afdfeb800c967ce2929bb85d797"}
Jul 28 20:14:34.288446 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.288402    2568 scope.go:117] "RemoveContainer" containerID="b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c"
Jul 28 20:14:34.289685 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.289664    2568 generic.go:356] "Generic (PLEG): container finished" podID="f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90" containerID="92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe" exitCode=0
Jul 28 20:14:34.289777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.289706    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"
Jul 28 20:14:34.289777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.289716    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x" event={"ID":"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90","Type":"ContainerDied","Data":"92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe"}
Jul 28 20:14:34.289777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.289759    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x" event={"ID":"f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90","Type":"ContainerDied","Data":"7ef8935482bcb746d6c4fcdf2b3d0dcd5de022d4021d314f00351735f7d38895"}
Jul 28 20:14:34.298994 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.298741    2568 scope.go:117] "RemoveContainer" containerID="b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c"
Jul 28 20:14:34.299306 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:14:34.299006    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c\": container with ID starting with b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c not found: ID does not exist" containerID="b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c"
Jul 28 20:14:34.299306 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.299029    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c"} err="failed to get container status \"b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c\": rpc error: code = NotFound desc = could not find container \"b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c\": container with ID starting with b1530d687af8b4e173848b0d1a3331ff1a5baeef2fe90837be2de5ec953b868c not found: ID does not exist"
Jul 28 20:14:34.299306 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.299045    2568 scope.go:117] "RemoveContainer" containerID="92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe"
Jul 28 20:14:34.307232 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.307211    2568 scope.go:117] "RemoveContainer" containerID="92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe"
Jul 28 20:14:34.307500 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:14:34.307478    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe\": container with ID starting with 92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe not found: ID does not exist" containerID="92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe"
Jul 28 20:14:34.307557 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.307506    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe"} err="failed to get container status \"92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe\": rpc error: code = NotFound desc = could not find container \"92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe\": container with ID starting with 92d742e76d47bc8bc890b4a44ff1803a96ca2ad461f422d93259a157cb4746fe not found: ID does not exist"
Jul 28 20:14:34.310763 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.310744    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"]
Jul 28 20:14:34.316118 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.316099    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ba3d8d-844fd56575-sdnwr"]
Jul 28 20:14:34.326885 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.326853    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"]
Jul 28 20:14:34.330967 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:34.330946    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-ba3d8d-c9979bffb-4g27x"]
Jul 28 20:14:35.676288 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:35.676256    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="daab92d3-acf8-4f8e-9d5c-095e5fcf0082" path="/var/lib/kubelet/pods/daab92d3-acf8-4f8e-9d5c-095e5fcf0082/volumes"
Jul 28 20:14:35.676661 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:35.676637    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90" path="/var/lib/kubelet/pods/f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90/volumes"
Jul 28 20:14:39.927898 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:39.927861    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:39.992763 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:39.992728    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:40.205628 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:40.205556    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:40.439157 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:40.439124    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:40.812194 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:40.812162    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:41.157701 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:41.157672    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:41.309518 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:41.309485    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:44.180175 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.180138    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:44.292201 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.292166    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:44.456288 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456199    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"]
Jul 28 20:14:44.456649 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456630    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90" containerName="payload-processing"
Jul 28 20:14:44.456740 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456651    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90" containerName="payload-processing"
Jul 28 20:14:44.456740 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456665    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="daab92d3-acf8-4f8e-9d5c-095e5fcf0082" containerName="maas-api"
Jul 28 20:14:44.456740 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456673    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="daab92d3-acf8-4f8e-9d5c-095e5fcf0082" containerName="maas-api"
Jul 28 20:14:44.456740 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456716    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2476b7c-63f3-466f-8c08-065d96b3b3f6" containerName="revoke-keys"
Jul 28 20:14:44.456740 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456725    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2476b7c-63f3-466f-8c08-065d96b3b3f6" containerName="revoke-keys"
Jul 28 20:14:44.456988 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456820    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="daab92d3-acf8-4f8e-9d5c-095e5fcf0082" containerName="maas-api"
Jul 28 20:14:44.456988 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456831    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2476b7c-63f3-466f-8c08-065d96b3b3f6" containerName="revoke-keys"
Jul 28 20:14:44.456988 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.456843    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6ffe1a7-6c8c-437a-bfac-3b6bf07beb90" containerName="payload-processing"
Jul 28 20:14:44.461633 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.461611    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:44.464539 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.464518    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:14:44.465244 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.465223    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:14:44.465244 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.465235    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-6e8ba4-serving-cert\""
Jul 28 20:14:44.465442 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.465238    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:14:44.472983 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.472954    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"]
Jul 28 20:14:44.511603 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.511573    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:44.554250 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.554222    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmgs6\" (UniqueName: \"kubernetes.io/projected/92fbe2c5-385c-445a-a595-c748f3de3ec5-kube-api-access-qmgs6\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:44.554402 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.554349    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fbe2c5-385c-445a-a595-c748f3de3ec5-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:44.554402 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.554385    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:44.655737 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.655694    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmgs6\" (UniqueName: \"kubernetes.io/projected/92fbe2c5-385c-445a-a595-c748f3de3ec5-kube-api-access-qmgs6\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:44.655938 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.655792    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fbe2c5-385c-445a-a595-c748f3de3ec5-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:44.655938 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.655815    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:44.655938 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:14:44.655915    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-serving-cert: secret "maas-api-e2e-api-a-6e8ba4-serving-cert" not found
Jul 28 20:14:44.656114 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:14:44.655984    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls podName:92fbe2c5-385c-445a-a595-c748f3de3ec5 nodeName:}" failed. No retries permitted until 2026-07-28 20:14:45.155966732 +0000 UTC m=+2397.967528593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls") pod "maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9" (UID: "92fbe2c5-385c-445a-a595-c748f3de3ec5") : secret "maas-api-e2e-api-a-6e8ba4-serving-cert" not found
Jul 28 20:14:44.656450 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.656426    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fbe2c5-385c-445a-a595-c748f3de3ec5-openshift-service-ca\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:44.665719 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:44.665692    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmgs6\" (UniqueName: \"kubernetes.io/projected/92fbe2c5-385c-445a-a595-c748f3de3ec5-kube-api-access-qmgs6\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:45.008602 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.008573    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:45.160160 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.160123    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:45.162592 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.162565    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls\") pod \"maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:45.326198 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.326118    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:45.373410 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.373376    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:45.442958 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.442919    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:14:45.526811 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.526779    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"]
Jul 28 20:14:45.528566 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.528544    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"]
Jul 28 20:14:45.533271 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:14:45.532210    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92fbe2c5_385c_445a_a595_c748f3de3ec5.slice/crio-6cbb32d4c3eb6cd6e12d78637b8877191ee8a7c001989e2b1254cbd0df8658b8 WatchSource:0}: Error finding container 6cbb32d4c3eb6cd6e12d78637b8877191ee8a7c001989e2b1254cbd0df8658b8: Status 404 returned error can't find the container with id 6cbb32d4c3eb6cd6e12d78637b8877191ee8a7c001989e2b1254cbd0df8658b8
Jul 28 20:14:45.534503 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.534473    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:45.540089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.540068    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-6e8ba4\""
Jul 28 20:14:45.540178 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.540160    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-6e8ba4-dockercfg-lcg82\""
Jul 28 20:14:45.554469 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.554443    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"]
Jul 28 20:14:45.562665 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.562644    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebfb137a-c4d3-42d4-b922-d5517e455227-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8\" (UID: \"ebfb137a-c4d3-42d4-b922-d5517e455227\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:45.562784 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.562767    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjc25\" (UniqueName: \"kubernetes.io/projected/ebfb137a-c4d3-42d4-b922-d5517e455227-kube-api-access-xjc25\") pod \"payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8\" (UID: \"ebfb137a-c4d3-42d4-b922-d5517e455227\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:45.644196 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.644166    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"]
Jul 28 20:14:45.647593 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.647576    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:45.664055 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.664031    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpdsq\" (UniqueName: \"kubernetes.io/projected/13dd0b51-f797-4e66-957c-8bd40fb97843-kube-api-access-mpdsq\") pod \"payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb\" (UID: \"13dd0b51-f797-4e66-957c-8bd40fb97843\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:45.664169 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.664073    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13dd0b51-f797-4e66-957c-8bd40fb97843-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb\" (UID: \"13dd0b51-f797-4e66-957c-8bd40fb97843\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:45.664169 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.664108    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjc25\" (UniqueName: \"kubernetes.io/projected/ebfb137a-c4d3-42d4-b922-d5517e455227-kube-api-access-xjc25\") pod \"payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8\" (UID: \"ebfb137a-c4d3-42d4-b922-d5517e455227\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:45.664286 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.664180    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebfb137a-c4d3-42d4-b922-d5517e455227-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8\" (UID: \"ebfb137a-c4d3-42d4-b922-d5517e455227\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:45.664793 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.664776    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebfb137a-c4d3-42d4-b922-d5517e455227-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8\" (UID: \"ebfb137a-c4d3-42d4-b922-d5517e455227\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:45.676720 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.676697    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"]
Jul 28 20:14:45.682360 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.682307    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjc25\" (UniqueName: \"kubernetes.io/projected/ebfb137a-c4d3-42d4-b922-d5517e455227-kube-api-access-xjc25\") pod \"payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8\" (UID: \"ebfb137a-c4d3-42d4-b922-d5517e455227\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:45.765205 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.765166    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpdsq\" (UniqueName: \"kubernetes.io/projected/13dd0b51-f797-4e66-957c-8bd40fb97843-kube-api-access-mpdsq\") pod \"payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb\" (UID: \"13dd0b51-f797-4e66-957c-8bd40fb97843\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:45.765430 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.765219    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13dd0b51-f797-4e66-957c-8bd40fb97843-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb\" (UID: \"13dd0b51-f797-4e66-957c-8bd40fb97843\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:45.765852 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.765832    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13dd0b51-f797-4e66-957c-8bd40fb97843-plugins-config-volume\") pod \"payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb\" (UID: \"13dd0b51-f797-4e66-957c-8bd40fb97843\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:45.774979 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.774953    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpdsq\" (UniqueName: \"kubernetes.io/projected/13dd0b51-f797-4e66-957c-8bd40fb97843-kube-api-access-mpdsq\") pod \"payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb\" (UID: \"13dd0b51-f797-4e66-957c-8bd40fb97843\") " pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:45.848084 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.848051    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:45.957451 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.957415    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:45.984958 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:45.984933    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"]
Jul 28 20:14:45.987294 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:14:45.987266    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfb137a_c4d3_42d4_b922_d5517e455227.slice/crio-c384e2fe8501bda0567df7d05d6dc09d14d1be35a8c2e347e292e51d64623246 WatchSource:0}: Error finding container c384e2fe8501bda0567df7d05d6dc09d14d1be35a8c2e347e292e51d64623246: Status 404 returned error can't find the container with id c384e2fe8501bda0567df7d05d6dc09d14d1be35a8c2e347e292e51d64623246
Jul 28 20:14:46.106639 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.106611    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"]
Jul 28 20:14:46.109626 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:14:46.109594    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13dd0b51_f797_4e66_957c_8bd40fb97843.slice/crio-92312d7aa934a9bf3322fe490a99a31fe105ddc4c7d4444ac0f710c5cbf55cfb WatchSource:0}: Error finding container 92312d7aa934a9bf3322fe490a99a31fe105ddc4c7d4444ac0f710c5cbf55cfb: Status 404 returned error can't find the container with id 92312d7aa934a9bf3322fe490a99a31fe105ddc4c7d4444ac0f710c5cbf55cfb
Jul 28 20:14:46.352601 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.352567    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9" event={"ID":"92fbe2c5-385c-445a-a595-c748f3de3ec5","Type":"ContainerStarted","Data":"00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4"}
Jul 28 20:14:46.353054 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.352632    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:46.353054 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.352656    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9" event={"ID":"92fbe2c5-385c-445a-a595-c748f3de3ec5","Type":"ContainerStarted","Data":"6cbb32d4c3eb6cd6e12d78637b8877191ee8a7c001989e2b1254cbd0df8658b8"}
Jul 28 20:14:46.354084 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.354062    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8" event={"ID":"ebfb137a-c4d3-42d4-b922-d5517e455227","Type":"ContainerStarted","Data":"9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b"}
Jul 28 20:14:46.354176 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.354100    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8" event={"ID":"ebfb137a-c4d3-42d4-b922-d5517e455227","Type":"ContainerStarted","Data":"c384e2fe8501bda0567df7d05d6dc09d14d1be35a8c2e347e292e51d64623246"}
Jul 28 20:14:46.354176 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.354160    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:14:46.355432 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.355411    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb" event={"ID":"13dd0b51-f797-4e66-957c-8bd40fb97843","Type":"ContainerStarted","Data":"914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3"}
Jul 28 20:14:46.355536 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.355453    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb" event={"ID":"13dd0b51-f797-4e66-957c-8bd40fb97843","Type":"ContainerStarted","Data":"92312d7aa934a9bf3322fe490a99a31fe105ddc4c7d4444ac0f710c5cbf55cfb"}
Jul 28 20:14:46.355536 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.355485    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:46.377242 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.377202    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9" podStartSLOduration=1.908626548 podStartE2EDuration="2.377188959s" totalImagesPullingTime="468.562411ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:14:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:14:46.376541432 +0000 UTC m=+2399.188103318" watchObservedRunningTime="2026-07-28 20:14:46.377188959 +0000 UTC m=+2399.188750823"
Jul 28 20:14:46.425277 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.425166    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb" podStartSLOduration=1.425149673 podStartE2EDuration="1.425149673s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:14:46.424976342 +0000 UTC m=+2399.236538235" watchObservedRunningTime="2026-07-28 20:14:46.425149673 +0000 UTC m=+2399.236711535"
Jul 28 20:14:46.425776 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:46.425738    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8" podStartSLOduration=1.425725644 podStartE2EDuration="1.425725644s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:14:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:14:46.401549765 +0000 UTC m=+2399.213111628" watchObservedRunningTime="2026-07-28 20:14:46.425725644 +0000 UTC m=+2399.237287507"
Jul 28 20:14:47.738071 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:47.738043    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:14:47.741402 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:47.741374    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:14:52.365603 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:52.365567    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:14:57.362387 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:57.362355    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:14:57.362847 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:14:57.362443    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:15:00.142830 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.142791    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"]
Jul 28 20:15:00.146360 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.146337    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng"]
Jul 28 20:15:00.146478 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.146437    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"
Jul 28 20:15:00.149231 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.149208    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:15:00.149718 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.149698    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng"
Jul 28 20:15:00.156977 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.156919    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"]
Jul 28 20:15:00.158731 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.158709    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"
Jul 28 20:15:00.159727 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.159702    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng"]
Jul 28 20:15:00.166748 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.166727    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng"
Jul 28 20:15:00.519106 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.519077    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng"]
Jul 28 20:15:00.521422 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:15:00.521392    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod953fcf3e_8065_4d9f_85c0_4e4b521ec81f.slice/crio-509774f36ac4e191e62fd851ddd42a36add9825f42d6fdb18ba2bf338ea60ae0 WatchSource:0}: Error finding container 509774f36ac4e191e62fd851ddd42a36add9825f42d6fdb18ba2bf338ea60ae0: Status 404 returned error can't find the container with id 509774f36ac4e191e62fd851ddd42a36add9825f42d6fdb18ba2bf338ea60ae0
Jul 28 20:15:00.539532 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:00.539402    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"]
Jul 28 20:15:00.541556 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:15:00.541534    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3725de16_dd80_42e5_a673_4d15f3cfe2b8.slice/crio-f1d6e0726c6b3652a1341f126b282c6235e2294bafa2820825bb36226ca5d351 WatchSource:0}: Error finding container f1d6e0726c6b3652a1341f126b282c6235e2294bafa2820825bb36226ca5d351: Status 404 returned error can't find the container with id f1d6e0726c6b3652a1341f126b282c6235e2294bafa2820825bb36226ca5d351
Jul 28 20:15:01.431964 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:01.431933    2568 generic.go:356] "Generic (PLEG): container finished" podID="3725de16-dd80-42e5-a673-4d15f3cfe2b8" containerID="e299cd5911b8af4bb0aa6df7c4a3e98f08d35174d4eadea0443b2446449d5a57" exitCode=0
Jul 28 20:15:01.432367 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:01.432023    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl" event={"ID":"3725de16-dd80-42e5-a673-4d15f3cfe2b8","Type":"ContainerDied","Data":"e299cd5911b8af4bb0aa6df7c4a3e98f08d35174d4eadea0443b2446449d5a57"}
Jul 28 20:15:01.432367 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:01.432089    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl" event={"ID":"3725de16-dd80-42e5-a673-4d15f3cfe2b8","Type":"ContainerStarted","Data":"f1d6e0726c6b3652a1341f126b282c6235e2294bafa2820825bb36226ca5d351"}
Jul 28 20:15:01.433596 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:01.433573    2568 generic.go:356] "Generic (PLEG): container finished" podID="953fcf3e-8065-4d9f-85c0-4e4b521ec81f" containerID="08fdc9860a9d8e595d443d05b5162d44344dbe67c148503466987990dc0a1a3a" exitCode=0
Jul 28 20:15:01.433699 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:01.433613    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng" event={"ID":"953fcf3e-8065-4d9f-85c0-4e4b521ec81f","Type":"ContainerDied","Data":"08fdc9860a9d8e595d443d05b5162d44344dbe67c148503466987990dc0a1a3a"}
Jul 28 20:15:01.433699 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:01.433655    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng" event={"ID":"953fcf3e-8065-4d9f-85c0-4e4b521ec81f","Type":"ContainerStarted","Data":"509774f36ac4e191e62fd851ddd42a36add9825f42d6fdb18ba2bf338ea60ae0"}
Jul 28 20:15:02.589086 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:02.589063    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"
Jul 28 20:15:02.609770 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:02.609743    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng"
Jul 28 20:15:03.443881 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:03.443851    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"
Jul 28 20:15:03.444045 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:03.443857    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl" event={"ID":"3725de16-dd80-42e5-a673-4d15f3cfe2b8","Type":"ContainerDied","Data":"f1d6e0726c6b3652a1341f126b282c6235e2294bafa2820825bb36226ca5d351"}
Jul 28 20:15:03.444045 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:03.443997    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f1d6e0726c6b3652a1341f126b282c6235e2294bafa2820825bb36226ca5d351"
Jul 28 20:15:03.445241 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:03.445223    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng"
Jul 28 20:15:03.445407 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:03.445268    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29754495-x4cng" event={"ID":"953fcf3e-8065-4d9f-85c0-4e4b521ec81f","Type":"ContainerDied","Data":"509774f36ac4e191e62fd851ddd42a36add9825f42d6fdb18ba2bf338ea60ae0"}
Jul 28 20:15:03.445407 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:03.445304    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="509774f36ac4e191e62fd851ddd42a36add9825f42d6fdb18ba2bf338ea60ae0"
Jul 28 20:15:09.246107 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:09.246064    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:09.762980 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:09.762949    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:10.236803 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:10.236774    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:10.319082 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:10.319046    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:13.383158 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.383112    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:13.468467 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.468425    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"]
Jul 28 20:15:13.468876 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.468859    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3725de16-dd80-42e5-a673-4d15f3cfe2b8" containerName="cleanup"
Jul 28 20:15:13.468951 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.468878    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="3725de16-dd80-42e5-a673-4d15f3cfe2b8" containerName="cleanup"
Jul 28 20:15:13.468951 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.468900    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="953fcf3e-8065-4d9f-85c0-4e4b521ec81f" containerName="cleanup"
Jul 28 20:15:13.468951 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.468910    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="953fcf3e-8065-4d9f-85c0-4e4b521ec81f" containerName="cleanup"
Jul 28 20:15:13.469059 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.469004    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="953fcf3e-8065-4d9f-85c0-4e4b521ec81f" containerName="cleanup"
Jul 28 20:15:13.469059 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.469021    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="3725de16-dd80-42e5-a673-4d15f3cfe2b8" containerName="cleanup"
Jul 28 20:15:13.470205 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.470175    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:13.472388 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.472369    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.474883 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.474860    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-0559d0-serving-cert\""
Jul 28 20:15:13.484640 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.484618    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"]
Jul 28 20:15:13.551040 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.551006    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:13.621166 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.621130    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c4373-6bb1-441a-a49d-e0edf616941f-openshift-service-ca\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.621360 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.621183    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c4373-6bb1-441a-a49d-e0edf616941f-maas-api-tls\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.621360 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.621267    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9q6mg\" (UniqueName: \"kubernetes.io/projected/444c4373-6bb1-441a-a49d-e0edf616941f-kube-api-access-9q6mg\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.702004 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.701965    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:13.722240 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.722207    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c4373-6bb1-441a-a49d-e0edf616941f-openshift-service-ca\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.722410 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.722250    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c4373-6bb1-441a-a49d-e0edf616941f-maas-api-tls\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.722410 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.722277    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9q6mg\" (UniqueName: \"kubernetes.io/projected/444c4373-6bb1-441a-a49d-e0edf616941f-kube-api-access-9q6mg\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.722924 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.722896    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c4373-6bb1-441a-a49d-e0edf616941f-openshift-service-ca\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.724841 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.724806    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c4373-6bb1-441a-a49d-e0edf616941f-maas-api-tls\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.732542 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.732514    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9q6mg\" (UniqueName: \"kubernetes.io/projected/444c4373-6bb1-441a-a49d-e0edf616941f-kube-api-access-9q6mg\") pod \"maas-api-e2e-api-b-0559d0-586b7b77-skdcm\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.783461 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.783431    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:13.943921 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:13.943895    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"]
Jul 28 20:15:13.945785 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:15:13.945754    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod444c4373_6bb1_441a_a49d_e0edf616941f.slice/crio-b40b8349423dbf76cd18c5daa452e1f878bea6416b8fc7663609f30b9494469a WatchSource:0}: Error finding container b40b8349423dbf76cd18c5daa452e1f878bea6416b8fc7663609f30b9494469a: Status 404 returned error can't find the container with id b40b8349423dbf76cd18c5daa452e1f878bea6416b8fc7663609f30b9494469a
Jul 28 20:15:14.077599 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.077562    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:14.170856 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.170826    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:14.203146 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.203113    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"]
Jul 28 20:15:14.208159 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.208088    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:14.210996 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.210838    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-0559d0-dockercfg-vng6z\""
Jul 28 20:15:14.211171 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.211145    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-0559d0\""
Jul 28 20:15:14.214071 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.214048    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"]
Jul 28 20:15:14.329532 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.329499    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-798cj\" (UniqueName: \"kubernetes.io/projected/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-kube-api-access-798cj\") pod \"payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6\" (UID: \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\") " pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:14.329713 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.329569    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6\" (UID: \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\") " pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:14.430375 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.430344    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-798cj\" (UniqueName: \"kubernetes.io/projected/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-kube-api-access-798cj\") pod \"payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6\" (UID: \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\") " pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:14.430742 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.430435    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6\" (UID: \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\") " pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:14.431084 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.431064    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-plugins-config-volume\") pod \"payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6\" (UID: \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\") " pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:14.438636 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.438615    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-798cj\" (UniqueName: \"kubernetes.io/projected/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-kube-api-access-798cj\") pod \"payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6\" (UID: \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\") " pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:14.503777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.503690    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm" event={"ID":"444c4373-6bb1-441a-a49d-e0edf616941f","Type":"ContainerStarted","Data":"81ebc0345e53bfd2013ffd8e8a32645a3afe278ae86af057e59f61a249cabcdb"}
Jul 28 20:15:14.503777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.503749    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm" event={"ID":"444c4373-6bb1-441a-a49d-e0edf616941f","Type":"ContainerStarted","Data":"b40b8349423dbf76cd18c5daa452e1f878bea6416b8fc7663609f30b9494469a"}
Jul 28 20:15:14.503958 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.503801    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:14.520156 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.520130    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:14.522744 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.522696    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm" podStartSLOduration=1.068655217 podStartE2EDuration="1.52268169s" totalImagesPullingTime="454.026473ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:14.52034588 +0000 UTC m=+2427.331907754" watchObservedRunningTime="2026-07-28 20:15:14.52268169 +0000 UTC m=+2427.334243552"
Jul 28 20:15:14.871038 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:14.871006    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"]
Jul 28 20:15:14.873313 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:15:14.873278    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64bd1c18_e5ac_442d_9a88_498d3a2cc9e8.slice/crio-0e47239610584461997bf37211d2c2f9cc9680b979b4f4ded9ff9786ee8ea3e9 WatchSource:0}: Error finding container 0e47239610584461997bf37211d2c2f9cc9680b979b4f4ded9ff9786ee8ea3e9: Status 404 returned error can't find the container with id 0e47239610584461997bf37211d2c2f9cc9680b979b4f4ded9ff9786ee8ea3e9
Jul 28 20:15:15.510337 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:15.510283    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6" event={"ID":"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8","Type":"ContainerStarted","Data":"d02010ae4a83d8f58b789aece5cb8246e410620c52db29ff2c200191527e05ac"}
Jul 28 20:15:15.510777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:15.510512    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:15.510777 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:15.510557    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6" event={"ID":"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8","Type":"ContainerStarted","Data":"0e47239610584461997bf37211d2c2f9cc9680b979b4f4ded9ff9786ee8ea3e9"}
Jul 28 20:15:15.528214 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:15.528154    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6" podStartSLOduration=1.528137578 podStartE2EDuration="1.528137578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:15:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:15:15.526267671 +0000 UTC m=+2428.337829551" watchObservedRunningTime="2026-07-28 20:15:15.528137578 +0000 UTC m=+2428.339699441"
Jul 28 20:15:20.516246 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:20.516212    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:15:26.518038 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:26.518006    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:15:38.447947 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:38.447913    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:47.941499 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:47.941465    2568 scope.go:117] "RemoveContainer" containerID="55034fd414accc78e18de7fb7dce8d1f924e62caef46b202b1e83fcf730bce65"
Jul 28 20:15:57.094802 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.094764    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"]
Jul 28 20:15:57.095334 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.095025    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9" podUID="92fbe2c5-385c-445a-a595-c748f3de3ec5" containerName="maas-api" containerID="cri-o://00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4" gracePeriod=30
Jul 28 20:15:57.123034 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.123006    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"]
Jul 28 20:15:57.125533 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.125509    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-a-6e8ba4-29754495-l25xl"]
Jul 28 20:15:57.127506 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.127451    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:57.153395 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.153367    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"]
Jul 28 20:15:57.153595 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.153570    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb" podUID="13dd0b51-f797-4e66-957c-8bd40fb97843" containerName="payload-processing" containerID="cri-o://914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3" gracePeriod=30
Jul 28 20:15:57.163472 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.163443    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"]
Jul 28 20:15:57.163712 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.163685    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8" podUID="ebfb137a-c4d3-42d4-b922-d5517e455227" containerName="payload-pre-processing" containerID="cri-o://9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b" gracePeriod=30
Jul 28 20:15:57.194910 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.194883    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:57.295439 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.295400    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:57.361051 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.360978    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8" podUID="ebfb137a-c4d3-42d4-b922-d5517e455227" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.70:9004: connect: connection refused"
Jul 28 20:15:57.397368 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.396804    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:57.422092 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.422067    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:15:57.444922 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.444841    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:15:57.456486 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.456462    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:15:57.493168 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493133    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpdsq\" (UniqueName: \"kubernetes.io/projected/13dd0b51-f797-4e66-957c-8bd40fb97843-kube-api-access-mpdsq\") pod \"13dd0b51-f797-4e66-957c-8bd40fb97843\" (UID: \"13dd0b51-f797-4e66-957c-8bd40fb97843\") "
Jul 28 20:15:57.493306 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493218    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fbe2c5-385c-445a-a595-c748f3de3ec5-openshift-service-ca\") pod \"92fbe2c5-385c-445a-a595-c748f3de3ec5\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") "
Jul 28 20:15:57.493306 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493265    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xjc25\" (UniqueName: \"kubernetes.io/projected/ebfb137a-c4d3-42d4-b922-d5517e455227-kube-api-access-xjc25\") pod \"ebfb137a-c4d3-42d4-b922-d5517e455227\" (UID: \"ebfb137a-c4d3-42d4-b922-d5517e455227\") "
Jul 28 20:15:57.493423 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493312    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls\") pod \"92fbe2c5-385c-445a-a595-c748f3de3ec5\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") "
Jul 28 20:15:57.493423 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493369    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qmgs6\" (UniqueName: \"kubernetes.io/projected/92fbe2c5-385c-445a-a595-c748f3de3ec5-kube-api-access-qmgs6\") pod \"92fbe2c5-385c-445a-a595-c748f3de3ec5\" (UID: \"92fbe2c5-385c-445a-a595-c748f3de3ec5\") "
Jul 28 20:15:57.493542 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493437    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13dd0b51-f797-4e66-957c-8bd40fb97843-plugins-config-volume\") pod \"13dd0b51-f797-4e66-957c-8bd40fb97843\" (UID: \"13dd0b51-f797-4e66-957c-8bd40fb97843\") "
Jul 28 20:15:57.493542 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493487    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebfb137a-c4d3-42d4-b922-d5517e455227-plugins-config-volume\") pod \"ebfb137a-c4d3-42d4-b922-d5517e455227\" (UID: \"ebfb137a-c4d3-42d4-b922-d5517e455227\") "
Jul 28 20:15:57.493676 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493634    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/92fbe2c5-385c-445a-a595-c748f3de3ec5-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "92fbe2c5-385c-445a-a595-c748f3de3ec5" (UID: "92fbe2c5-385c-445a-a595-c748f3de3ec5"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:15:57.493976 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.493951    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/92fbe2c5-385c-445a-a595-c748f3de3ec5-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:15:57.494153 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.494113    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebfb137a-c4d3-42d4-b922-d5517e455227-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ebfb137a-c4d3-42d4-b922-d5517e455227" (UID: "ebfb137a-c4d3-42d4-b922-d5517e455227"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:15:57.494473 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.494430    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13dd0b51-f797-4e66-957c-8bd40fb97843-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "13dd0b51-f797-4e66-957c-8bd40fb97843" (UID: "13dd0b51-f797-4e66-957c-8bd40fb97843"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:15:57.495842 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.495812    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebfb137a-c4d3-42d4-b922-d5517e455227-kube-api-access-xjc25" (OuterVolumeSpecName: "kube-api-access-xjc25") pod "ebfb137a-c4d3-42d4-b922-d5517e455227" (UID: "ebfb137a-c4d3-42d4-b922-d5517e455227"). InnerVolumeSpecName "kube-api-access-xjc25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:15:57.496153 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.496131    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13dd0b51-f797-4e66-957c-8bd40fb97843-kube-api-access-mpdsq" (OuterVolumeSpecName: "kube-api-access-mpdsq") pod "13dd0b51-f797-4e66-957c-8bd40fb97843" (UID: "13dd0b51-f797-4e66-957c-8bd40fb97843"). InnerVolumeSpecName "kube-api-access-mpdsq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:15:57.496238 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.496173    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "92fbe2c5-385c-445a-a595-c748f3de3ec5" (UID: "92fbe2c5-385c-445a-a595-c748f3de3ec5"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:15:57.496611 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.496591    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/92fbe2c5-385c-445a-a595-c748f3de3ec5-kube-api-access-qmgs6" (OuterVolumeSpecName: "kube-api-access-qmgs6") pod "92fbe2c5-385c-445a-a595-c748f3de3ec5" (UID: "92fbe2c5-385c-445a-a595-c748f3de3ec5"). InnerVolumeSpecName "kube-api-access-qmgs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:15:57.595307 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.595271    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/13dd0b51-f797-4e66-957c-8bd40fb97843-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:15:57.595307 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.595299    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ebfb137a-c4d3-42d4-b922-d5517e455227-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:15:57.595307 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.595308    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpdsq\" (UniqueName: \"kubernetes.io/projected/13dd0b51-f797-4e66-957c-8bd40fb97843-kube-api-access-mpdsq\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:15:57.595559 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.595317    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xjc25\" (UniqueName: \"kubernetes.io/projected/ebfb137a-c4d3-42d4-b922-d5517e455227-kube-api-access-xjc25\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:15:57.595559 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.595351    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/92fbe2c5-385c-445a-a595-c748f3de3ec5-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:15:57.595559 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.595360    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qmgs6\" (UniqueName: \"kubernetes.io/projected/92fbe2c5-385c-445a-a595-c748f3de3ec5-kube-api-access-qmgs6\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:15:57.680886 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.680848    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3725de16-dd80-42e5-a673-4d15f3cfe2b8" path="/var/lib/kubelet/pods/3725de16-dd80-42e5-a673-4d15f3cfe2b8/volumes"
Jul 28 20:15:57.709953 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.709926    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:15:57.721359 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.721315    2568 generic.go:356] "Generic (PLEG): container finished" podID="92fbe2c5-385c-445a-a595-c748f3de3ec5" containerID="00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4" exitCode=0
Jul 28 20:15:57.721481 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.721378    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"
Jul 28 20:15:57.721481 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.721433    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9" event={"ID":"92fbe2c5-385c-445a-a595-c748f3de3ec5","Type":"ContainerDied","Data":"00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4"}
Jul 28 20:15:57.721602 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.721491    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9" event={"ID":"92fbe2c5-385c-445a-a595-c748f3de3ec5","Type":"ContainerDied","Data":"6cbb32d4c3eb6cd6e12d78637b8877191ee8a7c001989e2b1254cbd0df8658b8"}
Jul 28 20:15:57.721602 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.721522    2568 scope.go:117] "RemoveContainer" containerID="00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4"
Jul 28 20:15:57.722970 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.722948    2568 generic.go:356] "Generic (PLEG): container finished" podID="ebfb137a-c4d3-42d4-b922-d5517e455227" containerID="9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b" exitCode=0
Jul 28 20:15:57.723092 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.722999    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"
Jul 28 20:15:57.723092 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.723056    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8" event={"ID":"ebfb137a-c4d3-42d4-b922-d5517e455227","Type":"ContainerDied","Data":"9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b"}
Jul 28 20:15:57.723213 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.723123    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8" event={"ID":"ebfb137a-c4d3-42d4-b922-d5517e455227","Type":"ContainerDied","Data":"c384e2fe8501bda0567df7d05d6dc09d14d1be35a8c2e347e292e51d64623246"}
Jul 28 20:15:57.724690 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.724671    2568 generic.go:356] "Generic (PLEG): container finished" podID="13dd0b51-f797-4e66-957c-8bd40fb97843" containerID="914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3" exitCode=0
Jul 28 20:15:57.724822 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.724701    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb" event={"ID":"13dd0b51-f797-4e66-957c-8bd40fb97843","Type":"ContainerDied","Data":"914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3"}
Jul 28 20:15:57.724933 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.724853    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb" event={"ID":"13dd0b51-f797-4e66-957c-8bd40fb97843","Type":"ContainerDied","Data":"92312d7aa934a9bf3322fe490a99a31fe105ddc4c7d4444ac0f710c5cbf55cfb"}
Jul 28 20:15:57.724933 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.724738    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"
Jul 28 20:15:57.732271 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.732251    2568 scope.go:117] "RemoveContainer" containerID="00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4"
Jul 28 20:15:57.732573 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:15:57.732554    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4\": container with ID starting with 00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4 not found: ID does not exist" containerID="00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4"
Jul 28 20:15:57.732663 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.732592    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4"} err="failed to get container status \"00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4\": rpc error: code = NotFound desc = could not find container \"00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4\": container with ID starting with 00a310345c32242953da94e76f6173d03e7b16a7655ab309fef9f2457a3108e4 not found: ID does not exist"
Jul 28 20:15:57.732663 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.732615    2568 scope.go:117] "RemoveContainer" containerID="9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b"
Jul 28 20:15:57.740362 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.740339    2568 scope.go:117] "RemoveContainer" containerID="9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b"
Jul 28 20:15:57.740611 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:15:57.740592    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b\": container with ID starting with 9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b not found: ID does not exist" containerID="9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b"
Jul 28 20:15:57.740675 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.740618    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b"} err="failed to get container status \"9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b\": rpc error: code = NotFound desc = could not find container \"9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b\": container with ID starting with 9364ce0e30b3c03cff66051c7706fcc8e1c0980154085b7d5fc64fce4415ca1b not found: ID does not exist"
Jul 28 20:15:57.740675 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.740635    2568 scope.go:117] "RemoveContainer" containerID="914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3"
Jul 28 20:15:57.748387 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.748369    2568 scope.go:117] "RemoveContainer" containerID="914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3"
Jul 28 20:15:57.748648 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:15:57.748629    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3\": container with ID starting with 914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3 not found: ID does not exist" containerID="914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3"
Jul 28 20:15:57.748701 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.748654    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3"} err="failed to get container status \"914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3\": rpc error: code = NotFound desc = could not find container \"914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3\": container with ID starting with 914c9458796e14d3aea56df514bbc3425733b06f6e71aaa30ad5592f53d1f5d3 not found: ID does not exist"
Jul 28 20:15:57.749770 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.749752    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"]
Jul 28 20:15:57.753610 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.753589    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-6e8ba4-6b948dd9b-zwvg8"]
Jul 28 20:15:57.769220 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.769199    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"]
Jul 28 20:15:57.775846 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.775826    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb"]
Jul 28 20:15:57.787254 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.787235    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"]
Jul 28 20:15:57.792353 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:57.792316    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9"]
Jul 28 20:15:58.360969 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:58.360873    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-api-a-6e8ba4-6656c9dd8-bp8mb" podUID="13dd0b51-f797-4e66-957c-8bd40fb97843" containerName="payload-processing" probeResult="failure" output="dial tcp 10.132.0.71:9004: i/o timeout"
Jul 28 20:15:59.676618 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:59.676581    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13dd0b51-f797-4e66-957c-8bd40fb97843" path="/var/lib/kubelet/pods/13dd0b51-f797-4e66-957c-8bd40fb97843/volumes"
Jul 28 20:15:59.677169 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:59.677148    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="92fbe2c5-385c-445a-a595-c748f3de3ec5" path="/var/lib/kubelet/pods/92fbe2c5-385c-445a-a595-c748f3de3ec5/volumes"
Jul 28 20:15:59.677571 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:15:59.677551    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebfb137a-c4d3-42d4-b922-d5517e455227" path="/var/lib/kubelet/pods/ebfb137a-c4d3-42d4-b922-d5517e455227/volumes"
Jul 28 20:16:00.361468 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:00.361422    2568 prober.go:120] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-6e8ba4-6b4789bddf-w89b9" podUID="92fbe2c5-385c-445a-a595-c748f3de3ec5" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.69:8443/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)"
Jul 28 20:16:03.569583 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:03.569542    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:03.631466 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:03.631441    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:03.853233 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:03.853161    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:04.102049 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.102006    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:04.233263 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233228    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"]
Jul 28 20:16:04.233614 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233601    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebfb137a-c4d3-42d4-b922-d5517e455227" containerName="payload-pre-processing"
Jul 28 20:16:04.233658 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233616    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebfb137a-c4d3-42d4-b922-d5517e455227" containerName="payload-pre-processing"
Jul 28 20:16:04.233658 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233627    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="13dd0b51-f797-4e66-957c-8bd40fb97843" containerName="payload-processing"
Jul 28 20:16:04.233658 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233633    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="13dd0b51-f797-4e66-957c-8bd40fb97843" containerName="payload-processing"
Jul 28 20:16:04.233658 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233651    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="92fbe2c5-385c-445a-a595-c748f3de3ec5" containerName="maas-api"
Jul 28 20:16:04.233658 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233657    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="92fbe2c5-385c-445a-a595-c748f3de3ec5" containerName="maas-api"
Jul 28 20:16:04.233812 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233710    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="92fbe2c5-385c-445a-a595-c748f3de3ec5" containerName="maas-api"
Jul 28 20:16:04.233812 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233722    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebfb137a-c4d3-42d4-b922-d5517e455227" containerName="payload-pre-processing"
Jul 28 20:16:04.233812 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.233729    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="13dd0b51-f797-4e66-957c-8bd40fb97843" containerName="payload-processing"
Jul 28 20:16:04.238539 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.238519    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"
Jul 28 20:16:04.241380 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.241353    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:16:04.243114 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.243086    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"]
Jul 28 20:16:04.359217 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.359173    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfde3c54-187f-4004-aa06-1d6bb402c0c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr\" (UID: \"bfde3c54-187f-4004-aa06-1d6bb402c0c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"
Jul 28 20:16:04.460378 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.460339    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfde3c54-187f-4004-aa06-1d6bb402c0c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr\" (UID: \"bfde3c54-187f-4004-aa06-1d6bb402c0c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"
Jul 28 20:16:04.460977 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.460955    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfde3c54-187f-4004-aa06-1d6bb402c0c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr\" (UID: \"bfde3c54-187f-4004-aa06-1d6bb402c0c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"
Jul 28 20:16:04.551702 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.551608    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"
Jul 28 20:16:04.683952 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.683925    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"]
Jul 28 20:16:04.685993 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:16:04.685965    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfde3c54_187f_4004_aa06_1d6bb402c0c8.slice/crio-511ba3737a641e7303691cee693b032525e52c151fe9f59cef6ce03f9be2267d WatchSource:0}: Error finding container 511ba3737a641e7303691cee693b032525e52c151fe9f59cef6ce03f9be2267d: Status 404 returned error can't find the container with id 511ba3737a641e7303691cee693b032525e52c151fe9f59cef6ce03f9be2267d
Jul 28 20:16:04.764204 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:04.764168    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr" event={"ID":"bfde3c54-187f-4004-aa06-1d6bb402c0c8","Type":"ContainerStarted","Data":"511ba3737a641e7303691cee693b032525e52c151fe9f59cef6ce03f9be2267d"}
Jul 28 20:16:05.770955 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:05.770924    2568 generic.go:356] "Generic (PLEG): container finished" podID="bfde3c54-187f-4004-aa06-1d6bb402c0c8" containerID="c4d49b8c7fe7a159d1490ea96d3f034ba276348c82cd3c00606959fb5ab73726" exitCode=0
Jul 28 20:16:05.771337 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:05.771002    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr" event={"ID":"bfde3c54-187f-4004-aa06-1d6bb402c0c8","Type":"ContainerDied","Data":"c4d49b8c7fe7a159d1490ea96d3f034ba276348c82cd3c00606959fb5ab73726"}
Jul 28 20:16:06.912249 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:06.912215    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"
Jul 28 20:16:06.982848 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:06.982814    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfde3c54-187f-4004-aa06-1d6bb402c0c8-maas-api-service-ca\") pod \"bfde3c54-187f-4004-aa06-1d6bb402c0c8\" (UID: \"bfde3c54-187f-4004-aa06-1d6bb402c0c8\") "
Jul 28 20:16:06.983195 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:06.983171    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfde3c54-187f-4004-aa06-1d6bb402c0c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "bfde3c54-187f-4004-aa06-1d6bb402c0c8" (UID: "bfde3c54-187f-4004-aa06-1d6bb402c0c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:16:07.083727 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:07.083634    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/bfde3c54-187f-4004-aa06-1d6bb402c0c8-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:16:07.788081 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:07.788039    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr" event={"ID":"bfde3c54-187f-4004-aa06-1d6bb402c0c8","Type":"ContainerDied","Data":"511ba3737a641e7303691cee693b032525e52c151fe9f59cef6ce03f9be2267d"}
Jul 28 20:16:07.788081 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:07.788081    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"
Jul 28 20:16:07.788358 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:07.788114    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="511ba3737a641e7303691cee693b032525e52c151fe9f59cef6ce03f9be2267d"
Jul 28 20:16:13.598265 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.598232    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"]
Jul 28 20:16:13.598736 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.598595    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm" podUID="444c4373-6bb1-441a-a49d-e0edf616941f" containerName="maas-api" containerID="cri-o://81ebc0345e53bfd2013ffd8e8a32645a3afe278ae86af057e59f61a249cabcdb" gracePeriod=30
Jul 28 20:16:13.628147 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.628122    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:13.661942 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.661912    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"]
Jul 28 20:16:13.662664 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.662598    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6" podUID="64bd1c18-e5ac-442d-9a88-498d3a2cc9e8" containerName="payload-processing" containerID="cri-o://d02010ae4a83d8f58b789aece5cb8246e410620c52db29ff2c200191527e05ac" gracePeriod=30
Jul 28 20:16:13.729097 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.729065    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:13.810146 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.810092    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"]
Jul 28 20:16:13.810888 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.810805    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:13.812770 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.812683    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-0559d0-pr8vr"]
Jul 28 20:16:13.818309 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.818206    2568 generic.go:356] "Generic (PLEG): container finished" podID="444c4373-6bb1-441a-a49d-e0edf616941f" containerID="81ebc0345e53bfd2013ffd8e8a32645a3afe278ae86af057e59f61a249cabcdb" exitCode=0
Jul 28 20:16:13.818309 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.818234    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm" event={"ID":"444c4373-6bb1-441a-a49d-e0edf616941f","Type":"ContainerDied","Data":"81ebc0345e53bfd2013ffd8e8a32645a3afe278ae86af057e59f61a249cabcdb"}
Jul 28 20:16:13.820246 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.820154    2568 generic.go:356] "Generic (PLEG): container finished" podID="64bd1c18-e5ac-442d-9a88-498d3a2cc9e8" containerID="d02010ae4a83d8f58b789aece5cb8246e410620c52db29ff2c200191527e05ac" exitCode=0
Jul 28 20:16:13.820246 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.820190    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6" event={"ID":"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8","Type":"ContainerDied","Data":"d02010ae4a83d8f58b789aece5cb8246e410620c52db29ff2c200191527e05ac"}
Jul 28 20:16:13.882272 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.882238    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:13.927946 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.927925    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:16:13.953769 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:13.953747    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:16:14.045635 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.045601    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-plugins-config-volume\") pod \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\" (UID: \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\") "
Jul 28 20:16:14.045801 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.045675    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c4373-6bb1-441a-a49d-e0edf616941f-openshift-service-ca\") pod \"444c4373-6bb1-441a-a49d-e0edf616941f\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") "
Jul 28 20:16:14.045801 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.045729    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-798cj\" (UniqueName: \"kubernetes.io/projected/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-kube-api-access-798cj\") pod \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\" (UID: \"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8\") "
Jul 28 20:16:14.045801 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.045753    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c4373-6bb1-441a-a49d-e0edf616941f-maas-api-tls\") pod \"444c4373-6bb1-441a-a49d-e0edf616941f\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") "
Jul 28 20:16:14.045801 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.045780    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9q6mg\" (UniqueName: \"kubernetes.io/projected/444c4373-6bb1-441a-a49d-e0edf616941f-kube-api-access-9q6mg\") pod \"444c4373-6bb1-441a-a49d-e0edf616941f\" (UID: \"444c4373-6bb1-441a-a49d-e0edf616941f\") "
Jul 28 20:16:14.046031 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.046001    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "64bd1c18-e5ac-442d-9a88-498d3a2cc9e8" (UID: "64bd1c18-e5ac-442d-9a88-498d3a2cc9e8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:16:14.046090 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.046024    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/444c4373-6bb1-441a-a49d-e0edf616941f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "444c4373-6bb1-441a-a49d-e0edf616941f" (UID: "444c4373-6bb1-441a-a49d-e0edf616941f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:16:14.047890 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.047868    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-kube-api-access-798cj" (OuterVolumeSpecName: "kube-api-access-798cj") pod "64bd1c18-e5ac-442d-9a88-498d3a2cc9e8" (UID: "64bd1c18-e5ac-442d-9a88-498d3a2cc9e8"). InnerVolumeSpecName "kube-api-access-798cj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:16:14.047890 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.047882    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/444c4373-6bb1-441a-a49d-e0edf616941f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "444c4373-6bb1-441a-a49d-e0edf616941f" (UID: "444c4373-6bb1-441a-a49d-e0edf616941f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:16:14.048001 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.047885    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/444c4373-6bb1-441a-a49d-e0edf616941f-kube-api-access-9q6mg" (OuterVolumeSpecName: "kube-api-access-9q6mg") pod "444c4373-6bb1-441a-a49d-e0edf616941f" (UID: "444c4373-6bb1-441a-a49d-e0edf616941f"). InnerVolumeSpecName "kube-api-access-9q6mg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:16:14.147159 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.147126    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9q6mg\" (UniqueName: \"kubernetes.io/projected/444c4373-6bb1-441a-a49d-e0edf616941f-kube-api-access-9q6mg\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:16:14.147304 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.147171    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:16:14.147304 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.147185    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/444c4373-6bb1-441a-a49d-e0edf616941f-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:16:14.147304 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.147198    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-798cj\" (UniqueName: \"kubernetes.io/projected/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8-kube-api-access-798cj\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:16:14.147304 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.147210    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/444c4373-6bb1-441a-a49d-e0edf616941f-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:16:14.307372 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.307312    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:14.826608 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.826577    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"
Jul 28 20:16:14.827036 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.826629    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm" event={"ID":"444c4373-6bb1-441a-a49d-e0edf616941f","Type":"ContainerDied","Data":"b40b8349423dbf76cd18c5daa452e1f878bea6416b8fc7663609f30b9494469a"}
Jul 28 20:16:14.827036 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.826700    2568 scope.go:117] "RemoveContainer" containerID="81ebc0345e53bfd2013ffd8e8a32645a3afe278ae86af057e59f61a249cabcdb"
Jul 28 20:16:14.828125 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.828107    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"
Jul 28 20:16:14.828207 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.828119    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6" event={"ID":"64bd1c18-e5ac-442d-9a88-498d3a2cc9e8","Type":"ContainerDied","Data":"0e47239610584461997bf37211d2c2f9cc9680b979b4f4ded9ff9786ee8ea3e9"}
Jul 28 20:16:14.837776 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.837759    2568 scope.go:117] "RemoveContainer" containerID="d02010ae4a83d8f58b789aece5cb8246e410620c52db29ff2c200191527e05ac"
Jul 28 20:16:14.851295 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.851269    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"]
Jul 28 20:16:14.855346 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.855314    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-0559d0-586b7b77-skdcm"]
Jul 28 20:16:14.866110 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.866050    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"]
Jul 28 20:16:14.873182 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:14.873160    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-0559d0-98c57f99b-tf9t6"]
Jul 28 20:16:15.675642 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:15.675611    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="444c4373-6bb1-441a-a49d-e0edf616941f" path="/var/lib/kubelet/pods/444c4373-6bb1-441a-a49d-e0edf616941f/volumes"
Jul 28 20:16:15.675965 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:15.675952    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="64bd1c18-e5ac-442d-9a88-498d3a2cc9e8" path="/var/lib/kubelet/pods/64bd1c18-e5ac-442d-9a88-498d3a2cc9e8/volumes"
Jul 28 20:16:15.676239 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:15.676228    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfde3c54-187f-4004-aa06-1d6bb402c0c8" path="/var/lib/kubelet/pods/bfde3c54-187f-4004-aa06-1d6bb402c0c8/volumes"
Jul 28 20:16:19.958416 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:19.958380    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:20.055410 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:20.055339    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:20.239145 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:20.239032    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:20.480062 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:20.480031    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:20.857312 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:20.857276    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:22.128585 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:22.128554    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:23.781681 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.781640    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"]
Jul 28 20:16:23.782160 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782148    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="444c4373-6bb1-441a-a49d-e0edf616941f" containerName="maas-api"
Jul 28 20:16:23.782230 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782163    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="444c4373-6bb1-441a-a49d-e0edf616941f" containerName="maas-api"
Jul 28 20:16:23.782230 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782183    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfde3c54-187f-4004-aa06-1d6bb402c0c8" containerName="revoke-keys"
Jul 28 20:16:23.782230 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782192    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfde3c54-187f-4004-aa06-1d6bb402c0c8" containerName="revoke-keys"
Jul 28 20:16:23.782432 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782248    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="64bd1c18-e5ac-442d-9a88-498d3a2cc9e8" containerName="payload-processing"
Jul 28 20:16:23.782432 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782259    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="64bd1c18-e5ac-442d-9a88-498d3a2cc9e8" containerName="payload-processing"
Jul 28 20:16:23.782432 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782371    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfde3c54-187f-4004-aa06-1d6bb402c0c8" containerName="revoke-keys"
Jul 28 20:16:23.782432 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782382    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="64bd1c18-e5ac-442d-9a88-498d3a2cc9e8" containerName="payload-processing"
Jul 28 20:16:23.782432 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.782395    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="444c4373-6bb1-441a-a49d-e0edf616941f" containerName="maas-api"
Jul 28 20:16:23.785808 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.785784    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:23.788530 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.788507    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-8szs5\""
Jul 28 20:16:23.788666 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.788618    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-bd928b-serving-cert\""
Jul 28 20:16:23.788742 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.788713    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Jul 28 20:16:23.789635 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.789614    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Jul 28 20:16:23.792768 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.792742    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"]
Jul 28 20:16:23.794205 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.794178    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:23.908826 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.908791    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:23.928677 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.928638    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fff4h\" (UniqueName: \"kubernetes.io/projected/4df8ebb5-402d-4e21-8a65-0b98802d7280-kube-api-access-fff4h\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:23.928828 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.928694    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4df8ebb5-402d-4e21-8a65-0b98802d7280-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:23.928828 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:23.928744    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4df8ebb5-402d-4e21-8a65-0b98802d7280-maas-api-tls\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.006174 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.006141    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:24.029394 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.029363    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fff4h\" (UniqueName: \"kubernetes.io/projected/4df8ebb5-402d-4e21-8a65-0b98802d7280-kube-api-access-fff4h\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.029514 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.029401    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4df8ebb5-402d-4e21-8a65-0b98802d7280-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.029514 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.029445    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4df8ebb5-402d-4e21-8a65-0b98802d7280-maas-api-tls\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.030167 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.030141    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4df8ebb5-402d-4e21-8a65-0b98802d7280-openshift-service-ca\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.032039 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.031986    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4df8ebb5-402d-4e21-8a65-0b98802d7280-maas-api-tls\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.038819 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.038787    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fff4h\" (UniqueName: \"kubernetes.io/projected/4df8ebb5-402d-4e21-8a65-0b98802d7280-kube-api-access-fff4h\") pod \"maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.098604 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.098570    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.231839 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.231735    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"]
Jul 28 20:16:24.234484 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:16:24.234455    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4df8ebb5_402d_4e21_8a65_0b98802d7280.slice/crio-255d016e27c970acf445801dc39c1ab331197188c76464fb9c33a9c60c899e57 WatchSource:0}: Error finding container 255d016e27c970acf445801dc39c1ab331197188c76464fb9c33a9c60c899e57: Status 404 returned error can't find the container with id 255d016e27c970acf445801dc39c1ab331197188c76464fb9c33a9c60c899e57
Jul 28 20:16:24.601591 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.601555    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:24.648855 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.648812    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"]
Jul 28 20:16:24.653158 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.653125    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:24.657403 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.657006    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-bd928b\""
Jul 28 20:16:24.671409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.671379    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"]
Jul 28 20:16:24.698070 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.698043    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"]
Jul 28 20:16:24.702545 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.702528    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:24.705477 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.705432    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-bd928b-dockercfg-xrq96\""
Jul 28 20:16:24.727257 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.727146    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"]
Jul 28 20:16:24.759418 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.759393    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:24.836157 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.836115    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-77tvf\" (UniqueName: \"kubernetes.io/projected/39a60e14-597c-40d0-aba1-d84514bb0ad0-kube-api-access-77tvf\") pod \"payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw\" (UID: \"39a60e14-597c-40d0-aba1-d84514bb0ad0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:24.836618 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.836173    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/216502f8-b4cf-492d-af3b-254ad380b2ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7\" (UID: \"216502f8-b4cf-492d-af3b-254ad380b2ae\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:24.836618 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.836201    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39a60e14-597c-40d0-aba1-d84514bb0ad0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw\" (UID: \"39a60e14-597c-40d0-aba1-d84514bb0ad0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:24.836618 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.836306    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kshl2\" (UniqueName: \"kubernetes.io/projected/216502f8-b4cf-492d-af3b-254ad380b2ae-kube-api-access-kshl2\") pod \"payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7\" (UID: \"216502f8-b4cf-492d-af3b-254ad380b2ae\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:24.881457 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.881359    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9" event={"ID":"4df8ebb5-402d-4e21-8a65-0b98802d7280","Type":"ContainerStarted","Data":"7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9"}
Jul 28 20:16:24.881457 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.881418    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9" event={"ID":"4df8ebb5-402d-4e21-8a65-0b98802d7280","Type":"ContainerStarted","Data":"255d016e27c970acf445801dc39c1ab331197188c76464fb9c33a9c60c899e57"}
Jul 28 20:16:24.881457 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.881440    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:24.899644 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.899594    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9" podStartSLOduration=1.46157122 podStartE2EDuration="1.899577747s" totalImagesPullingTime="438.006527ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:24.898781071 +0000 UTC m=+2497.710342954" watchObservedRunningTime="2026-07-28 20:16:24.899577747 +0000 UTC m=+2497.711139611"
Jul 28 20:16:24.937365 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.937305    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-77tvf\" (UniqueName: \"kubernetes.io/projected/39a60e14-597c-40d0-aba1-d84514bb0ad0-kube-api-access-77tvf\") pod \"payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw\" (UID: \"39a60e14-597c-40d0-aba1-d84514bb0ad0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:24.937570 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.937369    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/216502f8-b4cf-492d-af3b-254ad380b2ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7\" (UID: \"216502f8-b4cf-492d-af3b-254ad380b2ae\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:24.937570 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.937417    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39a60e14-597c-40d0-aba1-d84514bb0ad0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw\" (UID: \"39a60e14-597c-40d0-aba1-d84514bb0ad0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:24.937570 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.937477    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kshl2\" (UniqueName: \"kubernetes.io/projected/216502f8-b4cf-492d-af3b-254ad380b2ae-kube-api-access-kshl2\") pod \"payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7\" (UID: \"216502f8-b4cf-492d-af3b-254ad380b2ae\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:24.938112 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.938092    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/216502f8-b4cf-492d-af3b-254ad380b2ae-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7\" (UID: \"216502f8-b4cf-492d-af3b-254ad380b2ae\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:24.938191 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.938167    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39a60e14-597c-40d0-aba1-d84514bb0ad0-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw\" (UID: \"39a60e14-597c-40d0-aba1-d84514bb0ad0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:24.948559 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.948537    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-77tvf\" (UniqueName: \"kubernetes.io/projected/39a60e14-597c-40d0-aba1-d84514bb0ad0-kube-api-access-77tvf\") pod \"payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw\" (UID: \"39a60e14-597c-40d0-aba1-d84514bb0ad0\") " pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:24.948707 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.948633    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kshl2\" (UniqueName: \"kubernetes.io/projected/216502f8-b4cf-492d-af3b-254ad380b2ae-kube-api-access-kshl2\") pod \"payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7\" (UID: \"216502f8-b4cf-492d-af3b-254ad380b2ae\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:24.965954 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:24.965931    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:25.027195 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.027161    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:25.179595 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.179570    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"]
Jul 28 20:16:25.182404 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:16:25.182376    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216502f8_b4cf_492d_af3b_254ad380b2ae.slice/crio-596e7b7e10104dd9d0737197dcdc55f782e20e7c06976b8408cbc64d102293aa WatchSource:0}: Error finding container 596e7b7e10104dd9d0737197dcdc55f782e20e7c06976b8408cbc64d102293aa: Status 404 returned error can't find the container with id 596e7b7e10104dd9d0737197dcdc55f782e20e7c06976b8408cbc64d102293aa
Jul 28 20:16:25.325228 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.325202    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"]
Jul 28 20:16:25.327149 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:16:25.327119    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod39a60e14_597c_40d0_aba1_d84514bb0ad0.slice/crio-dfb0e53ea3beadc129084ee0c5adba8e21b68f588ae6f49bc17dcadb0f4e660a WatchSource:0}: Error finding container dfb0e53ea3beadc129084ee0c5adba8e21b68f588ae6f49bc17dcadb0f4e660a: Status 404 returned error can't find the container with id dfb0e53ea3beadc129084ee0c5adba8e21b68f588ae6f49bc17dcadb0f4e660a
Jul 28 20:16:25.889026 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.888985    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7" event={"ID":"216502f8-b4cf-492d-af3b-254ad380b2ae","Type":"ContainerStarted","Data":"6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7"}
Jul 28 20:16:25.889528 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.889050    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7" event={"ID":"216502f8-b4cf-492d-af3b-254ad380b2ae","Type":"ContainerStarted","Data":"596e7b7e10104dd9d0737197dcdc55f782e20e7c06976b8408cbc64d102293aa"}
Jul 28 20:16:25.889528 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.889183    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:25.890741 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.890711    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw" event={"ID":"39a60e14-597c-40d0-aba1-d84514bb0ad0","Type":"ContainerStarted","Data":"06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44"}
Jul 28 20:16:25.890885 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.890764    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw" event={"ID":"39a60e14-597c-40d0-aba1-d84514bb0ad0","Type":"ContainerStarted","Data":"dfb0e53ea3beadc129084ee0c5adba8e21b68f588ae6f49bc17dcadb0f4e660a"}
Jul 28 20:16:25.890885 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.890791    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:25.909699 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.909656    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7" podStartSLOduration=1.909643257 podStartE2EDuration="1.909643257s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:25.909103395 +0000 UTC m=+2498.720665260" watchObservedRunningTime="2026-07-28 20:16:25.909643257 +0000 UTC m=+2498.721205121"
Jul 28 20:16:25.929243 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:25.929198    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw" podStartSLOduration=1.929184324 podStartE2EDuration="1.929184324s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:25.927910348 +0000 UTC m=+2498.739472231" watchObservedRunningTime="2026-07-28 20:16:25.929184324 +0000 UTC m=+2498.740746192"
Jul 28 20:16:30.896056 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:30.896021    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:16:36.898032 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:36.897999    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:16:36.898520 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:36.898215    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:16:38.910597 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:38.910567    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:38.972909 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:38.972862    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:48.766847 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:48.766812    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:49.144602 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:49.144570    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:49.485030 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:49.484951    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:49.639286 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:49.639255    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:52.967729 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:52.967690    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:53.050414 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.050376    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:53.225278 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.225192    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"]
Jul 28 20:16:53.231026 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.231003    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.233998 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.233976    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-60efb4-serving-cert\""
Jul 28 20:16:53.236280 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.236260    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"]
Jul 28 20:16:53.254113 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.254078    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:53.294570 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.294542    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af859f3f-6b25-40f0-9d9c-f68729cf342c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.294744 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.294587    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvcqb\" (UniqueName: \"kubernetes.io/projected/af859f3f-6b25-40f0-9d9c-f68729cf342c-kube-api-access-wvcqb\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.294744 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.294616    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.396080 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.396037    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.396252 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.396149    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af859f3f-6b25-40f0-9d9c-f68729cf342c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.396252 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.396183    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wvcqb\" (UniqueName: \"kubernetes.io/projected/af859f3f-6b25-40f0-9d9c-f68729cf342c-kube-api-access-wvcqb\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.396252 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:16:53.396183    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-serving-cert: secret "maas-api-e2e-shared-b-60efb4-serving-cert" not found
Jul 28 20:16:53.396403 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:16:53.396288    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls podName:af859f3f-6b25-40f0-9d9c-f68729cf342c nodeName:}" failed. No retries permitted until 2026-07-28 20:16:53.896268285 +0000 UTC m=+2526.707830144 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls") pod "maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt" (UID: "af859f3f-6b25-40f0-9d9c-f68729cf342c") : secret "maas-api-e2e-shared-b-60efb4-serving-cert" not found
Jul 28 20:16:53.396819 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.396797    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af859f3f-6b25-40f0-9d9c-f68729cf342c-openshift-service-ca\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.406971 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.406942    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvcqb\" (UniqueName: \"kubernetes.io/projected/af859f3f-6b25-40f0-9d9c-f68729cf342c-kube-api-access-wvcqb\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.777823 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.777787    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:53.900756 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.900718    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.903100 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.903073    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls\") pod \"maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:53.905976 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.905956    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:16:53.910393 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.910370    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"]
Jul 28 20:16:53.914097 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.914078    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:53.917579 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.917561    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-60efb4\""
Jul 28 20:16:53.927849 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:53.927824    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"]
Jul 28 20:16:54.001725 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.001687    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zblrq\" (UniqueName: \"kubernetes.io/projected/7233b870-bc66-4d2d-9081-a4c6436f0b99-kube-api-access-zblrq\") pod \"payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc\" (UID: \"7233b870-bc66-4d2d-9081-a4c6436f0b99\") " pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:54.002111 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.001743    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7233b870-bc66-4d2d-9081-a4c6436f0b99-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc\" (UID: \"7233b870-bc66-4d2d-9081-a4c6436f0b99\") " pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:54.102713 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.102619    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zblrq\" (UniqueName: \"kubernetes.io/projected/7233b870-bc66-4d2d-9081-a4c6436f0b99-kube-api-access-zblrq\") pod \"payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc\" (UID: \"7233b870-bc66-4d2d-9081-a4c6436f0b99\") " pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:54.102713 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.102710    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7233b870-bc66-4d2d-9081-a4c6436f0b99-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc\" (UID: \"7233b870-bc66-4d2d-9081-a4c6436f0b99\") " pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:54.103463 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.103443    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7233b870-bc66-4d2d-9081-a4c6436f0b99-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc\" (UID: \"7233b870-bc66-4d2d-9081-a4c6436f0b99\") " pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:54.115970 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.115937    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zblrq\" (UniqueName: \"kubernetes.io/projected/7233b870-bc66-4d2d-9081-a4c6436f0b99-kube-api-access-zblrq\") pod \"payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc\" (UID: \"7233b870-bc66-4d2d-9081-a4c6436f0b99\") " pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:54.143909 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.143884    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:54.224306 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.224271    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:54.500115 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.500084    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"]
Jul 28 20:16:54.503481 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:16:54.503453    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf859f3f_6b25_40f0_9d9c_f68729cf342c.slice/crio-2edd99d1e8b8963672cf31750415c4d7b77ada6c361f9f754c8dff99d03cb9bd WatchSource:0}: Error finding container 2edd99d1e8b8963672cf31750415c4d7b77ada6c361f9f754c8dff99d03cb9bd: Status 404 returned error can't find the container with id 2edd99d1e8b8963672cf31750415c4d7b77ada6c361f9f754c8dff99d03cb9bd
Jul 28 20:16:54.587720 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:54.587687    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"]
Jul 28 20:16:54.592640 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:16:54.592611    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7233b870_bc66_4d2d_9081_a4c6436f0b99.slice/crio-bf3ce007b3a8832b5f21c54d6db06a252151e8643dc6f5feb7cfcd1e954a1362 WatchSource:0}: Error finding container bf3ce007b3a8832b5f21c54d6db06a252151e8643dc6f5feb7cfcd1e954a1362: Status 404 returned error can't find the container with id bf3ce007b3a8832b5f21c54d6db06a252151e8643dc6f5feb7cfcd1e954a1362
Jul 28 20:16:55.038393 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:55.038355    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc" event={"ID":"7233b870-bc66-4d2d-9081-a4c6436f0b99","Type":"ContainerStarted","Data":"06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5"}
Jul 28 20:16:55.038867 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:55.038423    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc" event={"ID":"7233b870-bc66-4d2d-9081-a4c6436f0b99","Type":"ContainerStarted","Data":"bf3ce007b3a8832b5f21c54d6db06a252151e8643dc6f5feb7cfcd1e954a1362"}
Jul 28 20:16:55.038867 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:55.038451    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:16:55.039683 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:55.039658    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt" event={"ID":"af859f3f-6b25-40f0-9d9c-f68729cf342c","Type":"ContainerStarted","Data":"2edd99d1e8b8963672cf31750415c4d7b77ada6c361f9f754c8dff99d03cb9bd"}
Jul 28 20:16:55.070744 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:55.070692    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc" podStartSLOduration=2.070674475 podStartE2EDuration="2.070674475s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:55.070437356 +0000 UTC m=+2527.881999253" watchObservedRunningTime="2026-07-28 20:16:55.070674475 +0000 UTC m=+2527.882236338"
Jul 28 20:16:56.046190 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:56.046147    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt" event={"ID":"af859f3f-6b25-40f0-9d9c-f68729cf342c","Type":"ContainerStarted","Data":"6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27"}
Jul 28 20:16:56.046625 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:56.046390    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:16:56.065579 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:16:56.065529    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt" podStartSLOduration=2.380291671 podStartE2EDuration="3.065515446s" totalImagesPullingTime="685.223775ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:16:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:16:56.063474021 +0000 UTC m=+2528.875035911" watchObservedRunningTime="2026-07-28 20:16:56.065515446 +0000 UTC m=+2528.877077309"
Jul 28 20:17:02.057826 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:17:02.057789    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:17:06.048429 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:17:06.048396    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:17:18.012531 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:17:18.012498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:23.949314 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:17:23.949278    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:24.241693 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:17:24.241613    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:24.342937 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:17:24.342888    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:17:48.690686 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:17:48.690653    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:18:34.678128 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:18:34.678093    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:19:47.769237 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:19:47.769157    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:19:47.775036 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:19:47.775008    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:20:06.488734 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:20:06.488696    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:32.649710 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:20:32.649668    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:20:48.015743 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:20:48.015707    2568 scope.go:117] "RemoveContainer" containerID="4d5ae7546a104e2c6a1a07bc75428bb43740a88085f14aaba3179973692e5e48"
Jul 28 20:20:48.024210 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:20:48.024190    2568 scope.go:117] "RemoveContainer" containerID="5313fcf4947c31e121897c069c1d22380badfff1a69dd060d66381d0c24b1413"
Jul 28 20:21:48.048299 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:21:48.048265    2568 scope.go:117] "RemoveContainer" containerID="e299cd5911b8af4bb0aa6df7c4a3e98f08d35174d4eadea0443b2446449d5a57"
Jul 28 20:22:14.281993 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:22:14.281951    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:15.745791 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:22:15.745749    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:24.870160 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:22:24.870124    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:48.072360 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:22:48.072264    2568 scope.go:117] "RemoveContainer" containerID="c4d49b8c7fe7a159d1490ea96d3f034ba276348c82cd3c00606959fb5ab73726"
Jul 28 20:22:57.581496 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:22:57.581456    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:57.680175 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:22:57.680135    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:22:57.755426 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:22:57.755390    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:28.946843 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:24:28.946811    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:29.664218 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:24:29.664181    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:42.950053 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:24:42.950015    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:24:47.800756 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:24:47.800726    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:24:47.807901 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:24:47.807876    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:25:07.332175 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:25:07.332135    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:41.501942 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:25:41.501844    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:55.094767 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:25:55.094729    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:25:55.167983 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:25:55.167947    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:16.740963 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:16.740916    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:17.664578 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:17.664541    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:37.585959 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:37.585921    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:37.935602 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:37.935565    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:38.587513 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:38.587470    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:40.988016 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:40.987953    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:41.080586 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:41.080543    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:41.235591 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:41.235558    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:41.519039 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:41.519002    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:42.050122 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.050072    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:42.143406 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.143366    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:26:42.261230 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.261190    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"]
Jul 28 20:26:42.264830 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.264810    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.267734 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.267715    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-d7a9b9-dockercfg-86hkz\""
Jul 28 20:26:42.267851 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.267814    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-d7a9b9\""
Jul 28 20:26:42.274678 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.274651    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"]
Jul 28 20:26:42.336255 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.336136    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/319aef48-0e66-4d6a-9dc5-e432e27f33a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb\" (UID: \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.336255 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.336223    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmlvn\" (UniqueName: \"kubernetes.io/projected/319aef48-0e66-4d6a-9dc5-e432e27f33a8-kube-api-access-bmlvn\") pod \"payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb\" (UID: \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.351819 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.351776    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"]
Jul 28 20:26:42.355250 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.355233    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.366604 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.366573    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"]
Jul 28 20:26:42.437470 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.437438    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a788f941-6063-43cb-bb73-66b4bc5415b0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z\" (UID: \"a788f941-6063-43cb-bb73-66b4bc5415b0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.437684 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.437477    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdtp5\" (UniqueName: \"kubernetes.io/projected/a788f941-6063-43cb-bb73-66b4bc5415b0-kube-api-access-tdtp5\") pod \"payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z\" (UID: \"a788f941-6063-43cb-bb73-66b4bc5415b0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.437684 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.437523    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/319aef48-0e66-4d6a-9dc5-e432e27f33a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb\" (UID: \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.437684 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.437554    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmlvn\" (UniqueName: \"kubernetes.io/projected/319aef48-0e66-4d6a-9dc5-e432e27f33a8-kube-api-access-bmlvn\") pod \"payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb\" (UID: \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.438193 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.438172    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/319aef48-0e66-4d6a-9dc5-e432e27f33a8-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb\" (UID: \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.446780 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.446754    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmlvn\" (UniqueName: \"kubernetes.io/projected/319aef48-0e66-4d6a-9dc5-e432e27f33a8-kube-api-access-bmlvn\") pod \"payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb\" (UID: \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.538544 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.538504    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a788f941-6063-43cb-bb73-66b4bc5415b0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z\" (UID: \"a788f941-6063-43cb-bb73-66b4bc5415b0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.538749 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.538550    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdtp5\" (UniqueName: \"kubernetes.io/projected/a788f941-6063-43cb-bb73-66b4bc5415b0-kube-api-access-tdtp5\") pod \"payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z\" (UID: \"a788f941-6063-43cb-bb73-66b4bc5415b0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.539280 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.539252    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a788f941-6063-43cb-bb73-66b4bc5415b0-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z\" (UID: \"a788f941-6063-43cb-bb73-66b4bc5415b0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.547959 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.547929    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdtp5\" (UniqueName: \"kubernetes.io/projected/a788f941-6063-43cb-bb73-66b4bc5415b0-kube-api-access-tdtp5\") pod \"payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z\" (UID: \"a788f941-6063-43cb-bb73-66b4bc5415b0\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.576579 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.576552    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.665713 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.665607    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.729946 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.729900    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"]
Jul 28 20:26:42.827631 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.827608    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"]
Jul 28 20:26:42.830267 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:26:42.830244    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda788f941_6063_43cb_bb73_66b4bc5415b0.slice/crio-043d715ad29b4b2d007c93404180090e1f966a1d04eb7220ada521dfc166d488 WatchSource:0}: Error finding container 043d715ad29b4b2d007c93404180090e1f966a1d04eb7220ada521dfc166d488: Status 404 returned error can't find the container with id 043d715ad29b4b2d007c93404180090e1f966a1d04eb7220ada521dfc166d488
Jul 28 20:26:42.959809 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.959766    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb" event={"ID":"319aef48-0e66-4d6a-9dc5-e432e27f33a8","Type":"ContainerStarted","Data":"269b6b497eb073632e36b16dd844b842eaaad8796de64cfafef630f4a389b7cb"}
Jul 28 20:26:42.960004 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.959839    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb" event={"ID":"319aef48-0e66-4d6a-9dc5-e432e27f33a8","Type":"ContainerStarted","Data":"bcebbb3340c8ec880ffdfe614f0f3e682c772d50b90ece8f4acc8774dd33fa65"}
Jul 28 20:26:42.960004 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.959875    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:26:42.961282 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.961253    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z" event={"ID":"a788f941-6063-43cb-bb73-66b4bc5415b0","Type":"ContainerStarted","Data":"3e8eb2f9805141282e1a5b7e53ab83590ee9933f6ed6c6c6f5bd732f4d7a2ab7"}
Jul 28 20:26:42.961419 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.961301    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:42.961419 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.961342    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z" event={"ID":"a788f941-6063-43cb-bb73-66b4bc5415b0","Type":"ContainerStarted","Data":"043d715ad29b4b2d007c93404180090e1f966a1d04eb7220ada521dfc166d488"}
Jul 28 20:26:42.983204 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:42.983151    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb" podStartSLOduration=0.98313609 podStartE2EDuration="983.13609ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:42.982000013 +0000 UTC m=+3115.793561876" watchObservedRunningTime="2026-07-28 20:26:42.98313609 +0000 UTC m=+3115.794697954"
Jul 28 20:26:43.021251 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:43.021197    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z" podStartSLOduration=1.021182361 podStartE2EDuration="1.021182361s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:26:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:26:43.020743856 +0000 UTC m=+3115.832305720" watchObservedRunningTime="2026-07-28 20:26:43.021182361 +0000 UTC m=+3115.832744224"
Jul 28 20:26:53.968829 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:53.968794    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:26:53.971255 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:26:53.968865    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:27:06.028850 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:06.028810    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:06.771597 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:06.771557    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:07.130041 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:07.129951    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:07.452629 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:07.452591    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.106678 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.106594    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.193234 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.193194    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"]
Jul 28 20:27:10.196831 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.196807    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.199579 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.199548    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-b-f11e93-serving-cert\""
Jul 28 20:27:10.209956 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.209929    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.213463 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.213437    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"]
Jul 28 20:27:10.298589 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.298545    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.298795 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.298596    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.298795 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.298736    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fv7fd\" (UniqueName: \"kubernetes.io/projected/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-kube-api-access-fv7fd\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.309437 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.309408    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.399266 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.399230    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fv7fd\" (UniqueName: \"kubernetes.io/projected/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-kube-api-access-fv7fd\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.399451 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.399357    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.399451 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.399397    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.399535 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:27:10.399508    2568 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-serving-cert: secret "maas-api-e2e-ipp-b-f11e93-serving-cert" not found
Jul 28 20:27:10.399594 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:27:10.399583    2568 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls podName:e2850405-aa2b-4303-b5d6-9aba1cf2aeb2 nodeName:}" failed. No retries permitted until 2026-07-28 20:27:10.899565091 +0000 UTC m=+3143.711126933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls") pod "maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh" (UID: "e2850405-aa2b-4303-b5d6-9aba1cf2aeb2") : secret "maas-api-e2e-ipp-b-f11e93-serving-cert" not found
Jul 28 20:27:10.400082 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.400061    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-openshift-service-ca\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.408615 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.408586    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fv7fd\" (UniqueName: \"kubernetes.io/projected/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-kube-api-access-fv7fd\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.447504 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.447474    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:10.904537 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.904497    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.906932 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.906899    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls\") pod \"maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:10.925172 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:10.925143    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:11.020594 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.020559    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:11.053014 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.052981    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"]
Jul 28 20:27:11.056699 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.056677    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:11.059252 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.059231    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-f11e93\""
Jul 28 20:27:11.071206 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.071183    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"]
Jul 28 20:27:11.106202 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.106167    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rrtdp\" (UniqueName: \"kubernetes.io/projected/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-kube-api-access-rrtdp\") pod \"payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5\" (UID: \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:11.106405 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.106233    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5\" (UID: \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:11.107010 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.106986    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:11.206758 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.206723    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rrtdp\" (UniqueName: \"kubernetes.io/projected/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-kube-api-access-rrtdp\") pod \"payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5\" (UID: \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:11.206953 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.206818    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5\" (UID: \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:11.207565 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.207541    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5\" (UID: \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:11.215058 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.215026    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rrtdp\" (UniqueName: \"kubernetes.io/projected/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-kube-api-access-rrtdp\") pod \"payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5\" (UID: \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:11.250151 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.250125    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"]
Jul 28 20:27:11.252503 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:27:11.252477    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2850405_aa2b_4303_b5d6_9aba1cf2aeb2.slice/crio-70b235139eccb4d6927ccdb7dcaedeb254b9923a2b7e0e1b3495ecb43f57b859 WatchSource:0}: Error finding container 70b235139eccb4d6927ccdb7dcaedeb254b9923a2b7e0e1b3495ecb43f57b859: Status 404 returned error can't find the container with id 70b235139eccb4d6927ccdb7dcaedeb254b9923a2b7e0e1b3495ecb43f57b859
Jul 28 20:27:11.254357 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.254337    2568 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 28 20:27:11.367067 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.367028    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:11.717961 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:11.717940    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"]
Jul 28 20:27:11.719876 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:27:11.719850    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd6d9014_ecd5_41d5_8bf6_90c213a5412d.slice/crio-ffdb1144f2c985ba67516e65a873f032271733ae635704d784db9ab051cd6036 WatchSource:0}: Error finding container ffdb1144f2c985ba67516e65a873f032271733ae635704d784db9ab051cd6036: Status 404 returned error can't find the container with id ffdb1144f2c985ba67516e65a873f032271733ae635704d784db9ab051cd6036
Jul 28 20:27:12.102755 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:12.102651    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5" event={"ID":"fd6d9014-ecd5-41d5-8bf6-90c213a5412d","Type":"ContainerStarted","Data":"bdd88fa0cf359b639b333724a9d131ec9cc32958ba4fdf6657a4f468e59274c7"}
Jul 28 20:27:12.102755 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:12.102721    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5" event={"ID":"fd6d9014-ecd5-41d5-8bf6-90c213a5412d","Type":"ContainerStarted","Data":"ffdb1144f2c985ba67516e65a873f032271733ae635704d784db9ab051cd6036"}
Jul 28 20:27:12.102983 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:12.102857    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:12.104166 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:12.104142    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh" event={"ID":"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2","Type":"ContainerStarted","Data":"feac2c3269ec48c1367b45ad354e15bcd363b94005604f469d9f23457aa936dc"}
Jul 28 20:27:12.104301 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:12.104190    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:12.104301 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:12.104213    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh" event={"ID":"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2","Type":"ContainerStarted","Data":"70b235139eccb4d6927ccdb7dcaedeb254b9923a2b7e0e1b3495ecb43f57b859"}
Jul 28 20:27:12.121691 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:12.121641    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5" podStartSLOduration=1.121623351 podStartE2EDuration="1.121623351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:12.121176796 +0000 UTC m=+3144.932738685" watchObservedRunningTime="2026-07-28 20:27:12.121623351 +0000 UTC m=+3144.933185215"
Jul 28 20:27:12.139390 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:12.139349    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh" podStartSLOduration=1.848263673 podStartE2EDuration="2.139336213s" totalImagesPullingTime="291.07254ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:27:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:27:12.137991956 +0000 UTC m=+3144.949553819" watchObservedRunningTime="2026-07-28 20:27:12.139336213 +0000 UTC m=+3144.950898100"
Jul 28 20:27:18.115717 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:18.115688    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:27:23.111601 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:23.111555    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:27:35.193676 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:35.193638    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:27:42.367441 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:42.367402    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5_fd6d9014-ecd5-41d5-8bf6-90c213a5412d/payload-processing/0.log"
Jul 28 20:27:45.074912 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:27:45.074877    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:05.632446 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:05.632410    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:47.867020 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:47.866942    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z_a788f941-6063-43cb-bb73-66b4bc5415b0/payload-processing/0.log"
Jul 28 20:28:48.132867 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:48.132789    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5_fd6d9014-ecd5-41d5-8bf6-90c213a5412d/payload-processing/0.log"
Jul 28 20:28:48.457283 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:48.457237    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:48.951410 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:48.951372    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:49.112064 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:49.112031    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:51.703758 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:51.703721    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:51.869300 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:51.869266    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:52.018590 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:52.018483    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:52.200060 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:52.200020    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:52.987664 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:52.987630    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:53.199580 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.199550    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:28:53.274293 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.274209    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"]
Jul 28 20:28:53.278110 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.278093    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:53.283367 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.283311    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-635813\""
Jul 28 20:28:53.346357 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.346303    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsrwx\" (UniqueName: \"kubernetes.io/projected/06abbc8b-246b-4ab7-bd62-ca815311e1fc-kube-api-access-nsrwx\") pod \"payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct\" (UID: \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:53.346491 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.346443    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06abbc8b-246b-4ab7-bd62-ca815311e1fc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct\" (UID: \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:53.372184 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.372153    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"]
Jul 28 20:28:53.447503 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.447468    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsrwx\" (UniqueName: \"kubernetes.io/projected/06abbc8b-246b-4ab7-bd62-ca815311e1fc-kube-api-access-nsrwx\") pod \"payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct\" (UID: \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:53.447650 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.447569    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06abbc8b-246b-4ab7-bd62-ca815311e1fc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct\" (UID: \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:53.448216 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.448198    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06abbc8b-246b-4ab7-bd62-ca815311e1fc-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct\" (UID: \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:53.476261 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.476237    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsrwx\" (UniqueName: \"kubernetes.io/projected/06abbc8b-246b-4ab7-bd62-ca815311e1fc-kube-api-access-nsrwx\") pod \"payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct\" (UID: \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:53.588849 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.588777    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:53.733006 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:53.732983    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"]
Jul 28 20:28:53.735085 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:28:53.735056    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06abbc8b_246b_4ab7_bd62_ca815311e1fc.slice/crio-5bf8881863753436623ba36526e3b8aadb7d5b29869fbebdc4d4a1a5343d9f2e WatchSource:0}: Error finding container 5bf8881863753436623ba36526e3b8aadb7d5b29869fbebdc4d4a1a5343d9f2e: Status 404 returned error can't find the container with id 5bf8881863753436623ba36526e3b8aadb7d5b29869fbebdc4d4a1a5343d9f2e
Jul 28 20:28:54.630776 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:54.630737    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct" event={"ID":"06abbc8b-246b-4ab7-bd62-ca815311e1fc","Type":"ContainerStarted","Data":"d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab"}
Jul 28 20:28:54.631196 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:54.630796    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct" event={"ID":"06abbc8b-246b-4ab7-bd62-ca815311e1fc","Type":"ContainerStarted","Data":"5bf8881863753436623ba36526e3b8aadb7d5b29869fbebdc4d4a1a5343d9f2e"}
Jul 28 20:28:54.631196 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:54.630879    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:28:54.668099 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:28:54.668051    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct" podStartSLOduration=1.66803411 podStartE2EDuration="1.66803411s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:28:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:28:54.665720017 +0000 UTC m=+3247.477281901" watchObservedRunningTime="2026-07-28 20:28:54.66803411 +0000 UTC m=+3247.479595973"
Jul 28 20:29:05.638478 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:05.638440    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:29:16.736650 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:16.736592    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:17.682302 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:17.682271    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"]
Jul 28 20:29:17.685929 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:17.685911    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"
Jul 28 20:29:17.690536 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:17.690506    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:29:17.715427 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:17.715387    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"]
Jul 28 20:29:17.780114 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:17.780073    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b295880-4578-4445-89ad-e78e03ab563e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57\" (UID: \"7b295880-4578-4445-89ad-e78e03ab563e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"
Jul 28 20:29:17.880701 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:17.880656    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b295880-4578-4445-89ad-e78e03ab563e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57\" (UID: \"7b295880-4578-4445-89ad-e78e03ab563e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"
Jul 28 20:29:17.881351 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:17.881307    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b295880-4578-4445-89ad-e78e03ab563e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57\" (UID: \"7b295880-4578-4445-89ad-e78e03ab563e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"
Jul 28 20:29:17.996701 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:17.996610    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"
Jul 28 20:29:18.355402 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:18.355298    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"]
Jul 28 20:29:18.358623 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:29:18.358593    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b295880_4578_4445_89ad_e78e03ab563e.slice/crio-2357c2273b9653aa6d6247beae6a7140498e86d1c4169fc5d3a86a8e36b05750 WatchSource:0}: Error finding container 2357c2273b9653aa6d6247beae6a7140498e86d1c4169fc5d3a86a8e36b05750: Status 404 returned error can't find the container with id 2357c2273b9653aa6d6247beae6a7140498e86d1c4169fc5d3a86a8e36b05750
Jul 28 20:29:18.751984 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:18.751947    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57" event={"ID":"7b295880-4578-4445-89ad-e78e03ab563e","Type":"ContainerStarted","Data":"50f3b7efd67bb665062f0db52741911e7185ff504345e5c41b996bbe56ca23d3"}
Jul 28 20:29:18.752147 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:18.752010    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57" event={"ID":"7b295880-4578-4445-89ad-e78e03ab563e","Type":"ContainerStarted","Data":"2357c2273b9653aa6d6247beae6a7140498e86d1c4169fc5d3a86a8e36b05750"}
Jul 28 20:29:18.769628 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:18.769574    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57" podStartSLOduration=1.76955689 podStartE2EDuration="1.76955689s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:29:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:29:18.767605189 +0000 UTC m=+3271.579167074" watchObservedRunningTime="2026-07-28 20:29:18.76955689 +0000 UTC m=+3271.581118753"
Jul 28 20:29:19.758675 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:19.758642    2568 generic.go:356] "Generic (PLEG): container finished" podID="7b295880-4578-4445-89ad-e78e03ab563e" containerID="50f3b7efd67bb665062f0db52741911e7185ff504345e5c41b996bbe56ca23d3" exitCode=0
Jul 28 20:29:19.759148 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:19.758694    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57" event={"ID":"7b295880-4578-4445-89ad-e78e03ab563e","Type":"ContainerDied","Data":"50f3b7efd67bb665062f0db52741911e7185ff504345e5c41b996bbe56ca23d3"}
Jul 28 20:29:20.904664 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:20.904637    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"
Jul 28 20:29:21.012746 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:21.012715    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b295880-4578-4445-89ad-e78e03ab563e-maas-api-service-ca\") pod \"7b295880-4578-4445-89ad-e78e03ab563e\" (UID: \"7b295880-4578-4445-89ad-e78e03ab563e\") "
Jul 28 20:29:21.013110 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:21.013087    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b295880-4578-4445-89ad-e78e03ab563e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7b295880-4578-4445-89ad-e78e03ab563e" (UID: "7b295880-4578-4445-89ad-e78e03ab563e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:21.113730 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:21.113651    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7b295880-4578-4445-89ad-e78e03ab563e-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:29:21.770061 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:21.770035    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"
Jul 28 20:29:21.770061 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:21.770049    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57" event={"ID":"7b295880-4578-4445-89ad-e78e03ab563e","Type":"ContainerDied","Data":"2357c2273b9653aa6d6247beae6a7140498e86d1c4169fc5d3a86a8e36b05750"}
Jul 28 20:29:21.770277 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:21.770095    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2357c2273b9653aa6d6247beae6a7140498e86d1c4169fc5d3a86a8e36b05750"
Jul 28 20:29:22.870248 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:22.870216    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:22.923189 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:22.923158    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"]
Jul 28 20:29:22.923402 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:22.923377    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct" podUID="06abbc8b-246b-4ab7-bd62-ca815311e1fc" containerName="payload-processing" containerID="cri-o://d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab" gracePeriod=30
Jul 28 20:29:22.989963 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:22.989926    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:23.133084 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.132971    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:23.194492 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.194467    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:29:23.226752 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.226721    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"]
Jul 28 20:29:23.229985 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.229957    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-635813-cpd57"]
Jul 28 20:29:23.233753 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.233698    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:23.334133 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.334097    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsrwx\" (UniqueName: \"kubernetes.io/projected/06abbc8b-246b-4ab7-bd62-ca815311e1fc-kube-api-access-nsrwx\") pod \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\" (UID: \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\") "
Jul 28 20:29:23.334312 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.334168    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06abbc8b-246b-4ab7-bd62-ca815311e1fc-plugins-config-volume\") pod \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\" (UID: \"06abbc8b-246b-4ab7-bd62-ca815311e1fc\") "
Jul 28 20:29:23.334626 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.334601    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06abbc8b-246b-4ab7-bd62-ca815311e1fc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "06abbc8b-246b-4ab7-bd62-ca815311e1fc" (UID: "06abbc8b-246b-4ab7-bd62-ca815311e1fc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:23.336307 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.336280    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06abbc8b-246b-4ab7-bd62-ca815311e1fc-kube-api-access-nsrwx" (OuterVolumeSpecName: "kube-api-access-nsrwx") pod "06abbc8b-246b-4ab7-bd62-ca815311e1fc" (UID: "06abbc8b-246b-4ab7-bd62-ca815311e1fc"). InnerVolumeSpecName "kube-api-access-nsrwx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:23.435092 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.435045    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsrwx\" (UniqueName: \"kubernetes.io/projected/06abbc8b-246b-4ab7-bd62-ca815311e1fc-kube-api-access-nsrwx\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:29:23.435092 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.435088    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/06abbc8b-246b-4ab7-bd62-ca815311e1fc-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:29:23.624499 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.624461    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:23.678369 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.678310    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b295880-4578-4445-89ad-e78e03ab563e" path="/var/lib/kubelet/pods/7b295880-4578-4445-89ad-e78e03ab563e/volumes"
Jul 28 20:29:23.781927 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.781895    2568 generic.go:356] "Generic (PLEG): container finished" podID="06abbc8b-246b-4ab7-bd62-ca815311e1fc" containerID="d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab" exitCode=0
Jul 28 20:29:23.782098 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.781955    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"
Jul 28 20:29:23.782098 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.781976    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct" event={"ID":"06abbc8b-246b-4ab7-bd62-ca815311e1fc","Type":"ContainerDied","Data":"d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab"}
Jul 28 20:29:23.782098 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.782030    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct" event={"ID":"06abbc8b-246b-4ab7-bd62-ca815311e1fc","Type":"ContainerDied","Data":"5bf8881863753436623ba36526e3b8aadb7d5b29869fbebdc4d4a1a5343d9f2e"}
Jul 28 20:29:23.782098 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.782052    2568 scope.go:117] "RemoveContainer" containerID="d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab"
Jul 28 20:29:23.791281 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.791257    2568 scope.go:117] "RemoveContainer" containerID="d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab"
Jul 28 20:29:23.791641 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:29:23.791620    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab\": container with ID starting with d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab not found: ID does not exist" containerID="d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab"
Jul 28 20:29:23.791714 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.791648    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab"} err="failed to get container status \"d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab\": rpc error: code = NotFound desc = could not find container \"d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab\": container with ID starting with d6703cd8f27370c55c4593819a1f849c88e61b7b215d638614b3bb5aa473f1ab not found: ID does not exist"
Jul 28 20:29:23.803501 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.803473    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"]
Jul 28 20:29:23.811902 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:23.811883    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-635813-6f764fdd64-pwrct"]
Jul 28 20:29:25.676895 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:25.676857    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06abbc8b-246b-4ab7-bd62-ca815311e1fc" path="/var/lib/kubelet/pods/06abbc8b-246b-4ab7-bd62-ca815311e1fc/volumes"
Jul 28 20:29:29.506240 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:29.506208    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:29.638824 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:29.638789    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:29.827840 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:29.827746    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:30.090880 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:30.090793    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:47.832551 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:47.832521    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:29:47.840983 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:47.840957    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:29:49.786919 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:49.786887    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"]
Jul 28 20:29:49.787381 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:49.787095    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z" podUID="a788f941-6063-43cb-bb73-66b4bc5415b0" containerName="payload-processing" containerID="cri-o://3e8eb2f9805141282e1a5b7e53ab83590ee9933f6ed6c6c6f5bd732f4d7a2ab7" gracePeriod=30
Jul 28 20:29:49.798298 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:49.798255    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"]
Jul 28 20:29:49.799369 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:49.799063    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb" podUID="319aef48-0e66-4d6a-9dc5-e432e27f33a8" containerName="payload-pre-processing" containerID="cri-o://269b6b497eb073632e36b16dd844b842eaaad8796de64cfafef630f4a389b7cb" gracePeriod=30
Jul 28 20:29:49.931034 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:49.930998    2568 generic.go:356] "Generic (PLEG): container finished" podID="a788f941-6063-43cb-bb73-66b4bc5415b0" containerID="3e8eb2f9805141282e1a5b7e53ab83590ee9933f6ed6c6c6f5bd732f4d7a2ab7" exitCode=0
Jul 28 20:29:49.931195 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:49.931071    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z" event={"ID":"a788f941-6063-43cb-bb73-66b4bc5415b0","Type":"ContainerDied","Data":"3e8eb2f9805141282e1a5b7e53ab83590ee9933f6ed6c6c6f5bd732f4d7a2ab7"}
Jul 28 20:29:49.932903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:49.932878    2568 generic.go:356] "Generic (PLEG): container finished" podID="319aef48-0e66-4d6a-9dc5-e432e27f33a8" containerID="269b6b497eb073632e36b16dd844b842eaaad8796de64cfafef630f4a389b7cb" exitCode=0
Jul 28 20:29:49.933012 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:49.932949    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb" event={"ID":"319aef48-0e66-4d6a-9dc5-e432e27f33a8","Type":"ContainerDied","Data":"269b6b497eb073632e36b16dd844b842eaaad8796de64cfafef630f4a389b7cb"}
Jul 28 20:29:50.103127 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.103099    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:29:50.106089 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.106063    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:29:50.177260 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.177224    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bmlvn\" (UniqueName: \"kubernetes.io/projected/319aef48-0e66-4d6a-9dc5-e432e27f33a8-kube-api-access-bmlvn\") pod \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\" (UID: \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\") "
Jul 28 20:29:50.177466 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.177267    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdtp5\" (UniqueName: \"kubernetes.io/projected/a788f941-6063-43cb-bb73-66b4bc5415b0-kube-api-access-tdtp5\") pod \"a788f941-6063-43cb-bb73-66b4bc5415b0\" (UID: \"a788f941-6063-43cb-bb73-66b4bc5415b0\") "
Jul 28 20:29:50.177466 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.177302    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/319aef48-0e66-4d6a-9dc5-e432e27f33a8-plugins-config-volume\") pod \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\" (UID: \"319aef48-0e66-4d6a-9dc5-e432e27f33a8\") "
Jul 28 20:29:50.177594 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.177481    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a788f941-6063-43cb-bb73-66b4bc5415b0-plugins-config-volume\") pod \"a788f941-6063-43cb-bb73-66b4bc5415b0\" (UID: \"a788f941-6063-43cb-bb73-66b4bc5415b0\") "
Jul 28 20:29:50.177827 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.177801    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/319aef48-0e66-4d6a-9dc5-e432e27f33a8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "319aef48-0e66-4d6a-9dc5-e432e27f33a8" (UID: "319aef48-0e66-4d6a-9dc5-e432e27f33a8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:50.177920 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.177823    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a788f941-6063-43cb-bb73-66b4bc5415b0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a788f941-6063-43cb-bb73-66b4bc5415b0" (UID: "a788f941-6063-43cb-bb73-66b4bc5415b0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:29:50.179381 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.179354    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/319aef48-0e66-4d6a-9dc5-e432e27f33a8-kube-api-access-bmlvn" (OuterVolumeSpecName: "kube-api-access-bmlvn") pod "319aef48-0e66-4d6a-9dc5-e432e27f33a8" (UID: "319aef48-0e66-4d6a-9dc5-e432e27f33a8"). InnerVolumeSpecName "kube-api-access-bmlvn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:50.179480 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.179399    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a788f941-6063-43cb-bb73-66b4bc5415b0-kube-api-access-tdtp5" (OuterVolumeSpecName: "kube-api-access-tdtp5") pod "a788f941-6063-43cb-bb73-66b4bc5415b0" (UID: "a788f941-6063-43cb-bb73-66b4bc5415b0"). InnerVolumeSpecName "kube-api-access-tdtp5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:29:50.278217 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.278184    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a788f941-6063-43cb-bb73-66b4bc5415b0-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:29:50.278217 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.278211    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bmlvn\" (UniqueName: \"kubernetes.io/projected/319aef48-0e66-4d6a-9dc5-e432e27f33a8-kube-api-access-bmlvn\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:29:50.278217 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.278221    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdtp5\" (UniqueName: \"kubernetes.io/projected/a788f941-6063-43cb-bb73-66b4bc5415b0-kube-api-access-tdtp5\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:29:50.278482 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.278231    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/319aef48-0e66-4d6a-9dc5-e432e27f33a8-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:29:50.559188 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.559136    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:50.659773 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.659725    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:29:50.939763 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.939734    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"
Jul 28 20:29:50.940233 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.939774    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z" event={"ID":"a788f941-6063-43cb-bb73-66b4bc5415b0","Type":"ContainerDied","Data":"043d715ad29b4b2d007c93404180090e1f966a1d04eb7220ada521dfc166d488"}
Jul 28 20:29:50.940233 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.939842    2568 scope.go:117] "RemoveContainer" containerID="3e8eb2f9805141282e1a5b7e53ab83590ee9933f6ed6c6c6f5bd732f4d7a2ab7"
Jul 28 20:29:50.941184 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.941082    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb" event={"ID":"319aef48-0e66-4d6a-9dc5-e432e27f33a8","Type":"ContainerDied","Data":"bcebbb3340c8ec880ffdfe614f0f3e682c772d50b90ece8f4acc8774dd33fa65"}
Jul 28 20:29:50.941184 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.941105    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"
Jul 28 20:29:50.949799 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.949778    2568 scope.go:117] "RemoveContainer" containerID="269b6b497eb073632e36b16dd844b842eaaad8796de64cfafef630f4a389b7cb"
Jul 28 20:29:50.967516 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.967494    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"]
Jul 28 20:29:50.982684 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.982663    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-d7a9b9-55fcf9b8b6-c7f6z"]
Jul 28 20:29:50.997378 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:50.997355    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"]
Jul 28 20:29:51.003896 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:51.003876    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-d7a9b9-5547cbff7b-rnjlb"]
Jul 28 20:29:51.676210 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:51.676178    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="319aef48-0e66-4d6a-9dc5-e432e27f33a8" path="/var/lib/kubelet/pods/319aef48-0e66-4d6a-9dc5-e432e27f33a8/volumes"
Jul 28 20:29:51.676584 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:51.676570    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a788f941-6063-43cb-bb73-66b4bc5415b0" path="/var/lib/kubelet/pods/a788f941-6063-43cb-bb73-66b4bc5415b0/volumes"
Jul 28 20:29:55.438113 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:29:55.438080    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:31.693016 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:30:31.692940    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:35.030503 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:30:35.030471    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:38.818292 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:30:38.818252    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:30:40.107040 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:30:40.106999    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:04.690268 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:04.690231    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:04.783266 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:04.783227    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:04.993344 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:04.993231    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:05.248532 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:05.248442    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:06.679926 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:06.679893    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:06.752375 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:06.752308    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:16.107137 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.107103    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"]
Jul 28 20:31:16.107682 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.107308    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh" podUID="e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" containerName="maas-api" containerID="cri-o://feac2c3269ec48c1367b45ad354e15bcd363b94005604f469d9f23457aa936dc" gracePeriod=30
Jul 28 20:31:16.136104 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.136073    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:16.200172 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.200125    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"]
Jul 28 20:31:16.200463 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.200412    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5" podUID="fd6d9014-ecd5-41d5-8bf6-90c213a5412d" containerName="payload-processing" containerID="cri-o://bdd88fa0cf359b639b333724a9d131ec9cc32958ba4fdf6657a4f468e59274c7" gracePeriod=30
Jul 28 20:31:16.268727 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.268695    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:16.373868 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.373285    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:16.382831 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.382792    2568 generic.go:356] "Generic (PLEG): container finished" podID="fd6d9014-ecd5-41d5-8bf6-90c213a5412d" containerID="bdd88fa0cf359b639b333724a9d131ec9cc32958ba4fdf6657a4f468e59274c7" exitCode=0
Jul 28 20:31:16.383052 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.383003    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5" event={"ID":"fd6d9014-ecd5-41d5-8bf6-90c213a5412d","Type":"ContainerDied","Data":"bdd88fa0cf359b639b333724a9d131ec9cc32958ba4fdf6657a4f468e59274c7"}
Jul 28 20:31:16.389283 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.389128    2568 generic.go:356] "Generic (PLEG): container finished" podID="e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" containerID="feac2c3269ec48c1367b45ad354e15bcd363b94005604f469d9f23457aa936dc" exitCode=0
Jul 28 20:31:16.389283 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.389176    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh" event={"ID":"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2","Type":"ContainerDied","Data":"feac2c3269ec48c1367b45ad354e15bcd363b94005604f469d9f23457aa936dc"}
Jul 28 20:31:16.455398 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.455368    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:31:16.459692 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.459663    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:16.501833 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.501809    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:31:16.590856 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.590822    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rrtdp\" (UniqueName: \"kubernetes.io/projected/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-kube-api-access-rrtdp\") pod \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\" (UID: \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\") "
Jul 28 20:31:16.591052 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.590880    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls\") pod \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") "
Jul 28 20:31:16.591052 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.590903    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fv7fd\" (UniqueName: \"kubernetes.io/projected/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-kube-api-access-fv7fd\") pod \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") "
Jul 28 20:31:16.591052 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.590929    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-openshift-service-ca\") pod \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\" (UID: \"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2\") "
Jul 28 20:31:16.591052 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.590962    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-plugins-config-volume\") pod \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\" (UID: \"fd6d9014-ecd5-41d5-8bf6-90c213a5412d\") "
Jul 28 20:31:16.591578 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.591445    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd6d9014-ecd5-41d5-8bf6-90c213a5412d" (UID: "fd6d9014-ecd5-41d5-8bf6-90c213a5412d"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:16.591687 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.591604    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" (UID: "e2850405-aa2b-4303-b5d6-9aba1cf2aeb2"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:16.593082 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.593054    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-kube-api-access-rrtdp" (OuterVolumeSpecName: "kube-api-access-rrtdp") pod "fd6d9014-ecd5-41d5-8bf6-90c213a5412d" (UID: "fd6d9014-ecd5-41d5-8bf6-90c213a5412d"). InnerVolumeSpecName "kube-api-access-rrtdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:16.593520 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.593503    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" (UID: "e2850405-aa2b-4303-b5d6-9aba1cf2aeb2"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:31:16.593520 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.593508    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-kube-api-access-fv7fd" (OuterVolumeSpecName: "kube-api-access-fv7fd") pod "e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" (UID: "e2850405-aa2b-4303-b5d6-9aba1cf2aeb2"). InnerVolumeSpecName "kube-api-access-fv7fd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:16.692279 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.692249    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:16.692279 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.692280    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:16.692509 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.692295    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rrtdp\" (UniqueName: \"kubernetes.io/projected/fd6d9014-ecd5-41d5-8bf6-90c213a5412d-kube-api-access-rrtdp\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:16.692509 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.692309    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:16.692509 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.692317    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fv7fd\" (UniqueName: \"kubernetes.io/projected/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2-kube-api-access-fv7fd\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:16.887481 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:16.887424    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:17.395856 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.395829    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"
Jul 28 20:31:17.396355 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.395854    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh" event={"ID":"e2850405-aa2b-4303-b5d6-9aba1cf2aeb2","Type":"ContainerDied","Data":"70b235139eccb4d6927ccdb7dcaedeb254b9923a2b7e0e1b3495ecb43f57b859"}
Jul 28 20:31:17.396355 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.395924    2568 scope.go:117] "RemoveContainer" containerID="feac2c3269ec48c1367b45ad354e15bcd363b94005604f469d9f23457aa936dc"
Jul 28 20:31:17.397397 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.397222    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"
Jul 28 20:31:17.397397 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.397228    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5" event={"ID":"fd6d9014-ecd5-41d5-8bf6-90c213a5412d","Type":"ContainerDied","Data":"ffdb1144f2c985ba67516e65a873f032271733ae635704d784db9ab051cd6036"}
Jul 28 20:31:17.405460 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.405395    2568 scope.go:117] "RemoveContainer" containerID="bdd88fa0cf359b639b333724a9d131ec9cc32958ba4fdf6657a4f468e59274c7"
Jul 28 20:31:17.444576 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.444548    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"]
Jul 28 20:31:17.451495 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.451472    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-b-f11e93-6b566bb5fd-7l8vh"]
Jul 28 20:31:17.475257 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.475226    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"]
Jul 28 20:31:17.503070 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.503041    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-f11e93-77d7c5b9c4-bwnr5"]
Jul 28 20:31:17.676831 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.676750    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" path="/var/lib/kubelet/pods/e2850405-aa2b-4303-b5d6-9aba1cf2aeb2/volumes"
Jul 28 20:31:17.677112 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:17.677099    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd6d9014-ecd5-41d5-8bf6-90c213a5412d" path="/var/lib/kubelet/pods/fd6d9014-ecd5-41d5-8bf6-90c213a5412d/volumes"
Jul 28 20:31:23.735644 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:23.735603    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:23.912058 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:23.912021    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:24.038950 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.038856    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:24.194945 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.194912    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:24.314705 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.314626    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"]
Jul 28 20:31:24.315011 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.314999    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd6d9014-ecd5-41d5-8bf6-90c213a5412d" containerName="payload-processing"
Jul 28 20:31:24.315053 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315013    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd6d9014-ecd5-41d5-8bf6-90c213a5412d" containerName="payload-processing"
Jul 28 20:31:24.315053 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315038    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="319aef48-0e66-4d6a-9dc5-e432e27f33a8" containerName="payload-pre-processing"
Jul 28 20:31:24.315053 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315044    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="319aef48-0e66-4d6a-9dc5-e432e27f33a8" containerName="payload-pre-processing"
Jul 28 20:31:24.315149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315055    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a788f941-6063-43cb-bb73-66b4bc5415b0" containerName="payload-processing"
Jul 28 20:31:24.315149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315061    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="a788f941-6063-43cb-bb73-66b4bc5415b0" containerName="payload-processing"
Jul 28 20:31:24.315149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315068    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" containerName="maas-api"
Jul 28 20:31:24.315149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315073    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" containerName="maas-api"
Jul 28 20:31:24.315149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315079    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06abbc8b-246b-4ab7-bd62-ca815311e1fc" containerName="payload-processing"
Jul 28 20:31:24.315149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315084    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="06abbc8b-246b-4ab7-bd62-ca815311e1fc" containerName="payload-processing"
Jul 28 20:31:24.315149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315094    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b295880-4578-4445-89ad-e78e03ab563e" containerName="revoke-keys"
Jul 28 20:31:24.315149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315100    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b295880-4578-4445-89ad-e78e03ab563e" containerName="revoke-keys"
Jul 28 20:31:24.315409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315163    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd6d9014-ecd5-41d5-8bf6-90c213a5412d" containerName="payload-processing"
Jul 28 20:31:24.315409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315171    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="06abbc8b-246b-4ab7-bd62-ca815311e1fc" containerName="payload-processing"
Jul 28 20:31:24.315409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315178    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="319aef48-0e66-4d6a-9dc5-e432e27f33a8" containerName="payload-pre-processing"
Jul 28 20:31:24.315409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315184    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7b295880-4578-4445-89ad-e78e03ab563e" containerName="revoke-keys"
Jul 28 20:31:24.315409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315191    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="a788f941-6063-43cb-bb73-66b4bc5415b0" containerName="payload-processing"
Jul 28 20:31:24.315409 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.315198    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e2850405-aa2b-4303-b5d6-9aba1cf2aeb2" containerName="maas-api"
Jul 28 20:31:24.319656 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.319637    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"
Jul 28 20:31:24.327810 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.327784    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:31:24.338973 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.338948    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"]
Jul 28 20:31:24.460904 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.460870    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbac35ec-9551-443b-88cf-c47d4e04e743-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw\" (UID: \"dbac35ec-9551-443b-88cf-c47d4e04e743\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"
Jul 28 20:31:24.562336 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.562287    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbac35ec-9551-443b-88cf-c47d4e04e743-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw\" (UID: \"dbac35ec-9551-443b-88cf-c47d4e04e743\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"
Jul 28 20:31:24.562921 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.562899    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbac35ec-9551-443b-88cf-c47d4e04e743-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw\" (UID: \"dbac35ec-9551-443b-88cf-c47d4e04e743\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"
Jul 28 20:31:24.630523 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.630438    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"
Jul 28 20:31:24.785086 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:24.785059    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"]
Jul 28 20:31:24.786677 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:31:24.786657    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbac35ec_9551_443b_88cf_c47d4e04e743.slice/crio-e9b763ba2da42f6f3b40c6466f718be7e22cfd53927dfb8aeae1d46804fc744b WatchSource:0}: Error finding container e9b763ba2da42f6f3b40c6466f718be7e22cfd53927dfb8aeae1d46804fc744b: Status 404 returned error can't find the container with id e9b763ba2da42f6f3b40c6466f718be7e22cfd53927dfb8aeae1d46804fc744b
Jul 28 20:31:25.439005 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:25.438966    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw" event={"ID":"dbac35ec-9551-443b-88cf-c47d4e04e743","Type":"ContainerStarted","Data":"640a15bf2a351205b2191072d0f232ec4a802f56ffb2d77588f40a6ee817203e"}
Jul 28 20:31:25.439225 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:25.439029    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw" event={"ID":"dbac35ec-9551-443b-88cf-c47d4e04e743","Type":"ContainerStarted","Data":"e9b763ba2da42f6f3b40c6466f718be7e22cfd53927dfb8aeae1d46804fc744b"}
Jul 28 20:31:25.472229 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:25.472169    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw" podStartSLOduration=1.472151577 podStartE2EDuration="1.472151577s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-28 20:31:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:31:25.471657071 +0000 UTC m=+3398.283218961" watchObservedRunningTime="2026-07-28 20:31:25.472151577 +0000 UTC m=+3398.283713440"
Jul 28 20:31:26.445719 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:26.445682    2568 generic.go:356] "Generic (PLEG): container finished" podID="dbac35ec-9551-443b-88cf-c47d4e04e743" containerID="640a15bf2a351205b2191072d0f232ec4a802f56ffb2d77588f40a6ee817203e" exitCode=0
Jul 28 20:31:26.446109 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:26.445767    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw" event={"ID":"dbac35ec-9551-443b-88cf-c47d4e04e743","Type":"ContainerDied","Data":"640a15bf2a351205b2191072d0f232ec4a802f56ffb2d77588f40a6ee817203e"}
Jul 28 20:31:27.592204 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:27.592172    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"
Jul 28 20:31:27.692709 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:27.692676    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbac35ec-9551-443b-88cf-c47d4e04e743-maas-api-service-ca\") pod \"dbac35ec-9551-443b-88cf-c47d4e04e743\" (UID: \"dbac35ec-9551-443b-88cf-c47d4e04e743\") "
Jul 28 20:31:27.693002 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:27.692980    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbac35ec-9551-443b-88cf-c47d4e04e743-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "dbac35ec-9551-443b-88cf-c47d4e04e743" (UID: "dbac35ec-9551-443b-88cf-c47d4e04e743"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:27.793857 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:27.793745    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/dbac35ec-9551-443b-88cf-c47d4e04e743-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:28.457149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:28.457113    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"
Jul 28 20:31:28.457149 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:28.457138    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw" event={"ID":"dbac35ec-9551-443b-88cf-c47d4e04e743","Type":"ContainerDied","Data":"e9b763ba2da42f6f3b40c6466f718be7e22cfd53927dfb8aeae1d46804fc744b"}
Jul 28 20:31:28.457381 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:28.457207    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9b763ba2da42f6f3b40c6466f718be7e22cfd53927dfb8aeae1d46804fc744b"
Jul 28 20:31:35.033767 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.033734    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"]
Jul 28 20:31:35.034175 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.033962    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9" podUID="4df8ebb5-402d-4e21-8a65-0b98802d7280" containerName="maas-api" containerID="cri-o://7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9" gracePeriod=30
Jul 28 20:31:35.103461 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.103430    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:35.182496 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.182455    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"]
Jul 28 20:31:35.182742 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.182717    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw" podUID="39a60e14-597c-40d0-aba1-d84514bb0ad0" containerName="payload-processing" containerID="cri-o://06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44" gracePeriod=30
Jul 28 20:31:35.201721 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.201687    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"]
Jul 28 20:31:35.201967 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.201943    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7" podUID="216502f8-b4cf-492d-af3b-254ad380b2ae" containerName="payload-pre-processing" containerID="cri-o://6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7" gracePeriod=30
Jul 28 20:31:35.296258 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:31:35.296187    2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216502f8_b4cf_492d_af3b_254ad380b2ae.slice/crio-6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7.scope\": RecentStats: unable to find data in memory cache]"
Jul 28 20:31:35.296415 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:31:35.296184    2568 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod216502f8_b4cf_492d_af3b_254ad380b2ae.slice/crio-6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7.scope\": RecentStats: unable to find data in memory cache]"
Jul 28 20:31:35.304127 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.304092    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:35.336453 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.336418    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:31:35.355059 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.354942    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fff4h\" (UniqueName: \"kubernetes.io/projected/4df8ebb5-402d-4e21-8a65-0b98802d7280-kube-api-access-fff4h\") pod \"4df8ebb5-402d-4e21-8a65-0b98802d7280\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") "
Jul 28 20:31:35.355059 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.355016    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4df8ebb5-402d-4e21-8a65-0b98802d7280-openshift-service-ca\") pod \"4df8ebb5-402d-4e21-8a65-0b98802d7280\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") "
Jul 28 20:31:35.355059 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.355050    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4df8ebb5-402d-4e21-8a65-0b98802d7280-maas-api-tls\") pod \"4df8ebb5-402d-4e21-8a65-0b98802d7280\" (UID: \"4df8ebb5-402d-4e21-8a65-0b98802d7280\") "
Jul 28 20:31:35.355464 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.355425    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4df8ebb5-402d-4e21-8a65-0b98802d7280-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4df8ebb5-402d-4e21-8a65-0b98802d7280" (UID: "4df8ebb5-402d-4e21-8a65-0b98802d7280"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:35.357354 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.357312    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4df8ebb5-402d-4e21-8a65-0b98802d7280-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4df8ebb5-402d-4e21-8a65-0b98802d7280" (UID: "4df8ebb5-402d-4e21-8a65-0b98802d7280"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:31:35.358056 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.358020    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4df8ebb5-402d-4e21-8a65-0b98802d7280-kube-api-access-fff4h" (OuterVolumeSpecName: "kube-api-access-fff4h") pod "4df8ebb5-402d-4e21-8a65-0b98802d7280" (UID: "4df8ebb5-402d-4e21-8a65-0b98802d7280"). InnerVolumeSpecName "kube-api-access-fff4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:35.442749 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.442707    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:35.456030 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.455998    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4df8ebb5-402d-4e21-8a65-0b98802d7280-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:35.456030 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.456032    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4df8ebb5-402d-4e21-8a65-0b98802d7280-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:35.456235 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.456048    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fff4h\" (UniqueName: \"kubernetes.io/projected/4df8ebb5-402d-4e21-8a65-0b98802d7280-kube-api-access-fff4h\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:35.465532 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.465506    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:31:35.476547 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.476524    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:31:35.492704 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.492671    2568 generic.go:356] "Generic (PLEG): container finished" podID="4df8ebb5-402d-4e21-8a65-0b98802d7280" containerID="7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9" exitCode=0
Jul 28 20:31:35.492915 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.492730    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"
Jul 28 20:31:35.492915 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.492768    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9" event={"ID":"4df8ebb5-402d-4e21-8a65-0b98802d7280","Type":"ContainerDied","Data":"7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9"}
Jul 28 20:31:35.492915 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.492860    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9" event={"ID":"4df8ebb5-402d-4e21-8a65-0b98802d7280","Type":"ContainerDied","Data":"255d016e27c970acf445801dc39c1ab331197188c76464fb9c33a9c60c899e57"}
Jul 28 20:31:35.492915 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.492894    2568 scope.go:117] "RemoveContainer" containerID="7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9"
Jul 28 20:31:35.494259 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.494235    2568 generic.go:356] "Generic (PLEG): container finished" podID="216502f8-b4cf-492d-af3b-254ad380b2ae" containerID="6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7" exitCode=0
Jul 28 20:31:35.494366 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.494283    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"
Jul 28 20:31:35.494366 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.494317    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7" event={"ID":"216502f8-b4cf-492d-af3b-254ad380b2ae","Type":"ContainerDied","Data":"6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7"}
Jul 28 20:31:35.494490 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.494391    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7" event={"ID":"216502f8-b4cf-492d-af3b-254ad380b2ae","Type":"ContainerDied","Data":"596e7b7e10104dd9d0737197dcdc55f782e20e7c06976b8408cbc64d102293aa"}
Jul 28 20:31:35.495660 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.495638    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"]
Jul 28 20:31:35.495872 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.495850    2568 generic.go:356] "Generic (PLEG): container finished" podID="39a60e14-597c-40d0-aba1-d84514bb0ad0" containerID="06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44" exitCode=0
Jul 28 20:31:35.495929 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.495885    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"
Jul 28 20:31:35.495929 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.495903    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw" event={"ID":"39a60e14-597c-40d0-aba1-d84514bb0ad0","Type":"ContainerDied","Data":"06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44"}
Jul 28 20:31:35.496006 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.495944    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw" event={"ID":"39a60e14-597c-40d0-aba1-d84514bb0ad0","Type":"ContainerDied","Data":"dfb0e53ea3beadc129084ee0c5adba8e21b68f588ae6f49bc17dcadb0f4e660a"}
Jul 28 20:31:35.503727 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.503692    2568 scope.go:117] "RemoveContainer" containerID="7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9"
Jul 28 20:31:35.504206 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:31:35.504028    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9\": container with ID starting with 7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9 not found: ID does not exist" containerID="7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9"
Jul 28 20:31:35.504206 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.504086    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9"} err="failed to get container status \"7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9\": rpc error: code = NotFound desc = could not find container \"7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9\": container with ID starting with 7d9d8b831d2908253000c73b0439b34382f7d27360b9f3959c84998dbfafe9d9 not found: ID does not exist"
Jul 28 20:31:35.504206 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.504111    2568 scope.go:117] "RemoveContainer" containerID="6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7"
Jul 28 20:31:35.508176 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.508149    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-bd928b-glmnw"]
Jul 28 20:31:35.513453 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.513432    2568 scope.go:117] "RemoveContainer" containerID="6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7"
Jul 28 20:31:35.513730 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:31:35.513710    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7\": container with ID starting with 6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7 not found: ID does not exist" containerID="6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7"
Jul 28 20:31:35.513816 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.513738    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7"} err="failed to get container status \"6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7\": rpc error: code = NotFound desc = could not find container \"6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7\": container with ID starting with 6bf858f76aebbfd087445bac80234e8eb478eeb373c01cb8292dae0a4b07fdf7 not found: ID does not exist"
Jul 28 20:31:35.513816 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.513757    2568 scope.go:117] "RemoveContainer" containerID="06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44"
Jul 28 20:31:35.521862 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.521842    2568 scope.go:117] "RemoveContainer" containerID="06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44"
Jul 28 20:31:35.522154 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:31:35.522131    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44\": container with ID starting with 06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44 not found: ID does not exist" containerID="06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44"
Jul 28 20:31:35.522250 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.522160    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44"} err="failed to get container status \"06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44\": rpc error: code = NotFound desc = could not find container \"06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44\": container with ID starting with 06fd51f2b0c434bc94438fdad71c9046c8512c74a3684c6fcf7078a11222ec44 not found: ID does not exist"
Jul 28 20:31:35.556695 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.556601    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kshl2\" (UniqueName: \"kubernetes.io/projected/216502f8-b4cf-492d-af3b-254ad380b2ae-kube-api-access-kshl2\") pod \"216502f8-b4cf-492d-af3b-254ad380b2ae\" (UID: \"216502f8-b4cf-492d-af3b-254ad380b2ae\") "
Jul 28 20:31:35.556873 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.556709    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39a60e14-597c-40d0-aba1-d84514bb0ad0-plugins-config-volume\") pod \"39a60e14-597c-40d0-aba1-d84514bb0ad0\" (UID: \"39a60e14-597c-40d0-aba1-d84514bb0ad0\") "
Jul 28 20:31:35.556873 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.556746    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/216502f8-b4cf-492d-af3b-254ad380b2ae-plugins-config-volume\") pod \"216502f8-b4cf-492d-af3b-254ad380b2ae\" (UID: \"216502f8-b4cf-492d-af3b-254ad380b2ae\") "
Jul 28 20:31:35.557308 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.557268    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/39a60e14-597c-40d0-aba1-d84514bb0ad0-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "39a60e14-597c-40d0-aba1-d84514bb0ad0" (UID: "39a60e14-597c-40d0-aba1-d84514bb0ad0"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:35.557308 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.557300    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/216502f8-b4cf-492d-af3b-254ad380b2ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "216502f8-b4cf-492d-af3b-254ad380b2ae" (UID: "216502f8-b4cf-492d-af3b-254ad380b2ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:35.557941 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.557916    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:35.559034 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.559010    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/216502f8-b4cf-492d-af3b-254ad380b2ae-kube-api-access-kshl2" (OuterVolumeSpecName: "kube-api-access-kshl2") pod "216502f8-b4cf-492d-af3b-254ad380b2ae" (UID: "216502f8-b4cf-492d-af3b-254ad380b2ae"). InnerVolumeSpecName "kube-api-access-kshl2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:35.586175 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.586141    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"]
Jul 28 20:31:35.597454 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.597420    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-bd928b-56bb78589f-qtlr9"]
Jul 28 20:31:35.657915 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.657879    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-77tvf\" (UniqueName: \"kubernetes.io/projected/39a60e14-597c-40d0-aba1-d84514bb0ad0-kube-api-access-77tvf\") pod \"39a60e14-597c-40d0-aba1-d84514bb0ad0\" (UID: \"39a60e14-597c-40d0-aba1-d84514bb0ad0\") "
Jul 28 20:31:35.658100 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.658087    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kshl2\" (UniqueName: \"kubernetes.io/projected/216502f8-b4cf-492d-af3b-254ad380b2ae-kube-api-access-kshl2\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:35.658155 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.658102    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/39a60e14-597c-40d0-aba1-d84514bb0ad0-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:35.658155 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.658112    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/216502f8-b4cf-492d-af3b-254ad380b2ae-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:35.659899 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.659875    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39a60e14-597c-40d0-aba1-d84514bb0ad0-kube-api-access-77tvf" (OuterVolumeSpecName: "kube-api-access-77tvf") pod "39a60e14-597c-40d0-aba1-d84514bb0ad0" (UID: "39a60e14-597c-40d0-aba1-d84514bb0ad0"). InnerVolumeSpecName "kube-api-access-77tvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:35.676300 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.676271    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4df8ebb5-402d-4e21-8a65-0b98802d7280" path="/var/lib/kubelet/pods/4df8ebb5-402d-4e21-8a65-0b98802d7280/volumes"
Jul 28 20:31:35.676703 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.676682    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbac35ec-9551-443b-88cf-c47d4e04e743" path="/var/lib/kubelet/pods/dbac35ec-9551-443b-88cf-c47d4e04e743/volumes"
Jul 28 20:31:35.758673 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.758636    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-77tvf\" (UniqueName: \"kubernetes.io/projected/39a60e14-597c-40d0-aba1-d84514bb0ad0-kube-api-access-77tvf\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:35.822468 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.822387    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"]
Jul 28 20:31:35.835646 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.835614    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-bd928b-64f778ccb4-qg2j7"]
Jul 28 20:31:35.859649 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.859616    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"]
Jul 28 20:31:35.862603 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:35.862576    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-bd928b-66f5b9cfc4-rcxjw"]
Jul 28 20:31:36.022970 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:36.022935    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:37.676639 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:37.676599    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="216502f8-b4cf-492d-af3b-254ad380b2ae" path="/var/lib/kubelet/pods/216502f8-b4cf-492d-af3b-254ad380b2ae/volumes"
Jul 28 20:31:37.677136 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:37.677118    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39a60e14-597c-40d0-aba1-d84514bb0ad0" path="/var/lib/kubelet/pods/39a60e14-597c-40d0-aba1-d84514bb0ad0/volumes"
Jul 28 20:31:43.647784 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:43.647707    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:43.787095 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:43.787042    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:43.979114 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:43.979028    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:44.332514 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332436    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"]
Jul 28 20:31:44.332818 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332805    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="216502f8-b4cf-492d-af3b-254ad380b2ae" containerName="payload-pre-processing"
Jul 28 20:31:44.332864 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332821    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="216502f8-b4cf-492d-af3b-254ad380b2ae" containerName="payload-pre-processing"
Jul 28 20:31:44.332864 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332836    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbac35ec-9551-443b-88cf-c47d4e04e743" containerName="revoke-keys"
Jul 28 20:31:44.332864 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332842    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbac35ec-9551-443b-88cf-c47d4e04e743" containerName="revoke-keys"
Jul 28 20:31:44.332864 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332857    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4df8ebb5-402d-4e21-8a65-0b98802d7280" containerName="maas-api"
Jul 28 20:31:44.332990 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332865    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="4df8ebb5-402d-4e21-8a65-0b98802d7280" containerName="maas-api"
Jul 28 20:31:44.332990 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332874    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39a60e14-597c-40d0-aba1-d84514bb0ad0" containerName="payload-processing"
Jul 28 20:31:44.332990 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332880    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="39a60e14-597c-40d0-aba1-d84514bb0ad0" containerName="payload-processing"
Jul 28 20:31:44.332990 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332939    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="4df8ebb5-402d-4e21-8a65-0b98802d7280" containerName="maas-api"
Jul 28 20:31:44.332990 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332951    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbac35ec-9551-443b-88cf-c47d4e04e743" containerName="revoke-keys"
Jul 28 20:31:44.332990 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332960    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="216502f8-b4cf-492d-af3b-254ad380b2ae" containerName="payload-pre-processing"
Jul 28 20:31:44.332990 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.332967    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="39a60e14-597c-40d0-aba1-d84514bb0ad0" containerName="payload-processing"
Jul 28 20:31:44.337719 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.337698    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"
Jul 28 20:31:44.340646 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.340608    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-bcmt7\""
Jul 28 20:31:44.343141 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.343101    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"]
Jul 28 20:31:44.419111 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.419082    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5\" (UID: \"e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"
Jul 28 20:31:44.519650 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.519621    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5\" (UID: \"e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"
Jul 28 20:31:44.520229 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.520209    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5\" (UID: \"e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"
Jul 28 20:31:44.649842 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.649799    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"
Jul 28 20:31:44.732536 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.732498    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:44.790751 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:44.790723    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"]
Jul 28 20:31:44.793241 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:31:44.793204    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode658b90d_27d9_42b5_8bcd_6fa6f7e88d5c.slice/crio-51d39bf92b50f8edb17a53c37299da23edb9ce468a5a672151a61129f6e76e7b WatchSource:0}: Error finding container 51d39bf92b50f8edb17a53c37299da23edb9ce468a5a672151a61129f6e76e7b: Status 404 returned error can't find the container with id 51d39bf92b50f8edb17a53c37299da23edb9ce468a5a672151a61129f6e76e7b
Jul 28 20:31:45.551792 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:45.551760    2568 generic.go:356] "Generic (PLEG): container finished" podID="e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c" containerID="fb58c0993eeaa926ea44a460692cf0b9fb1c675b90da0333cea5f3314e9b1998" exitCode=0
Jul 28 20:31:45.551975 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:45.551835    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5" event={"ID":"e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c","Type":"ContainerDied","Data":"fb58c0993eeaa926ea44a460692cf0b9fb1c675b90da0333cea5f3314e9b1998"}
Jul 28 20:31:45.551975 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:45.551887    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5" event={"ID":"e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c","Type":"ContainerStarted","Data":"51d39bf92b50f8edb17a53c37299da23edb9ce468a5a672151a61129f6e76e7b"}
Jul 28 20:31:46.694002 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:46.693976    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"
Jul 28 20:31:46.736622 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:46.736591    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c-maas-api-service-ca\") pod \"e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c\" (UID: \"e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c\") "
Jul 28 20:31:46.736924 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:46.736898    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c" (UID: "e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:46.837550 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:46.837458    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c-maas-api-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:47.563082 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:47.563046    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5" event={"ID":"e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c","Type":"ContainerDied","Data":"51d39bf92b50f8edb17a53c37299da23edb9ce468a5a672151a61129f6e76e7b"}
Jul 28 20:31:47.563082 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:47.563072    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"
Jul 28 20:31:47.563390 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:47.563094    2568 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="51d39bf92b50f8edb17a53c37299da23edb9ce468a5a672151a61129f6e76e7b"
Jul 28 20:31:55.001607 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.001568    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"]
Jul 28 20:31:55.004172 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.001860    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt" podUID="af859f3f-6b25-40f0-9d9c-f68729cf342c" containerName="maas-api" containerID="cri-o://6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27" gracePeriod=30
Jul 28 20:31:55.040687 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.040655    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:55.071782 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.071738    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"]
Jul 28 20:31:55.072053 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.072000    2568 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc" podUID="7233b870-bc66-4d2d-9081-a4c6436f0b99" containerName="payload-processing" containerID="cri-o://06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5" gracePeriod=30
Jul 28 20:31:55.140791 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.140761    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:55.247255 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.247153    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:55.274565 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.274484    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"]
Jul 28 20:31:55.279244 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.279219    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-60efb4-sqcr5"]
Jul 28 20:31:55.323946 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.323921    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:31:55.351032 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.351009    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:31:55.371285 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.371253    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:55.415512 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.415483    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvcqb\" (UniqueName: \"kubernetes.io/projected/af859f3f-6b25-40f0-9d9c-f68729cf342c-kube-api-access-wvcqb\") pod \"af859f3f-6b25-40f0-9d9c-f68729cf342c\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") "
Jul 28 20:31:55.415679 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.415529    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls\") pod \"af859f3f-6b25-40f0-9d9c-f68729cf342c\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") "
Jul 28 20:31:55.415679 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.415552    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zblrq\" (UniqueName: \"kubernetes.io/projected/7233b870-bc66-4d2d-9081-a4c6436f0b99-kube-api-access-zblrq\") pod \"7233b870-bc66-4d2d-9081-a4c6436f0b99\" (UID: \"7233b870-bc66-4d2d-9081-a4c6436f0b99\") "
Jul 28 20:31:55.415679 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.415642    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7233b870-bc66-4d2d-9081-a4c6436f0b99-plugins-config-volume\") pod \"7233b870-bc66-4d2d-9081-a4c6436f0b99\" (UID: \"7233b870-bc66-4d2d-9081-a4c6436f0b99\") "
Jul 28 20:31:55.415864 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.415681    2568 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af859f3f-6b25-40f0-9d9c-f68729cf342c-openshift-service-ca\") pod \"af859f3f-6b25-40f0-9d9c-f68729cf342c\" (UID: \"af859f3f-6b25-40f0-9d9c-f68729cf342c\") "
Jul 28 20:31:55.416132 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.416102    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/af859f3f-6b25-40f0-9d9c-f68729cf342c-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "af859f3f-6b25-40f0-9d9c-f68729cf342c" (UID: "af859f3f-6b25-40f0-9d9c-f68729cf342c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:55.416132 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.416116    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7233b870-bc66-4d2d-9081-a4c6436f0b99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7233b870-bc66-4d2d-9081-a4c6436f0b99" (UID: "7233b870-bc66-4d2d-9081-a4c6436f0b99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 28 20:31:55.417897 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.417875    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "af859f3f-6b25-40f0-9d9c-f68729cf342c" (UID: "af859f3f-6b25-40f0-9d9c-f68729cf342c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 28 20:31:55.418197 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.418179    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/af859f3f-6b25-40f0-9d9c-f68729cf342c-kube-api-access-wvcqb" (OuterVolumeSpecName: "kube-api-access-wvcqb") pod "af859f3f-6b25-40f0-9d9c-f68729cf342c" (UID: "af859f3f-6b25-40f0-9d9c-f68729cf342c"). InnerVolumeSpecName "kube-api-access-wvcqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:55.418256 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.418197    2568 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7233b870-bc66-4d2d-9081-a4c6436f0b99-kube-api-access-zblrq" (OuterVolumeSpecName: "kube-api-access-zblrq") pod "7233b870-bc66-4d2d-9081-a4c6436f0b99" (UID: "7233b870-bc66-4d2d-9081-a4c6436f0b99"). InnerVolumeSpecName "kube-api-access-zblrq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 28 20:31:55.516339 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.516289    2568 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7233b870-bc66-4d2d-9081-a4c6436f0b99-plugins-config-volume\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:55.516513 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.516347    2568 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/af859f3f-6b25-40f0-9d9c-f68729cf342c-openshift-service-ca\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:55.516513 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.516361    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvcqb\" (UniqueName: \"kubernetes.io/projected/af859f3f-6b25-40f0-9d9c-f68729cf342c-kube-api-access-wvcqb\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:55.516513 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.516373    2568 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/af859f3f-6b25-40f0-9d9c-f68729cf342c-maas-api-tls\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:55.516513 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.516384    2568 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zblrq\" (UniqueName: \"kubernetes.io/projected/7233b870-bc66-4d2d-9081-a4c6436f0b99-kube-api-access-zblrq\") on node \"ip-10-0-128-255.ec2.internal\" DevicePath \"\""
Jul 28 20:31:55.607610 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.607521    2568 generic.go:356] "Generic (PLEG): container finished" podID="7233b870-bc66-4d2d-9081-a4c6436f0b99" containerID="06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5" exitCode=0
Jul 28 20:31:55.607610 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.607577    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"
Jul 28 20:31:55.607850 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.607656    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc" event={"ID":"7233b870-bc66-4d2d-9081-a4c6436f0b99","Type":"ContainerDied","Data":"06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5"}
Jul 28 20:31:55.607850 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.607737    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc" event={"ID":"7233b870-bc66-4d2d-9081-a4c6436f0b99","Type":"ContainerDied","Data":"bf3ce007b3a8832b5f21c54d6db06a252151e8643dc6f5feb7cfcd1e954a1362"}
Jul 28 20:31:55.607850 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.607768    2568 scope.go:117] "RemoveContainer" containerID="06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5"
Jul 28 20:31:55.609125 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.609100    2568 generic.go:356] "Generic (PLEG): container finished" podID="af859f3f-6b25-40f0-9d9c-f68729cf342c" containerID="6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27" exitCode=0
Jul 28 20:31:55.609220 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.609202    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt" event={"ID":"af859f3f-6b25-40f0-9d9c-f68729cf342c","Type":"ContainerDied","Data":"6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27"}
Jul 28 20:31:55.609271 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.609242    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt" event={"ID":"af859f3f-6b25-40f0-9d9c-f68729cf342c","Type":"ContainerDied","Data":"2edd99d1e8b8963672cf31750415c4d7b77ada6c361f9f754c8dff99d03cb9bd"}
Jul 28 20:31:55.609271 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.609252    2568 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"
Jul 28 20:31:55.618933 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.618912    2568 scope.go:117] "RemoveContainer" containerID="06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5"
Jul 28 20:31:55.619259 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:31:55.619232    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5\": container with ID starting with 06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5 not found: ID does not exist" containerID="06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5"
Jul 28 20:31:55.619363 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.619273    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5"} err="failed to get container status \"06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5\": rpc error: code = NotFound desc = could not find container \"06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5\": container with ID starting with 06929ca6453fcca32d2dd88c865955e45f6ce687931014b79d471b90bfebe6e5 not found: ID does not exist"
Jul 28 20:31:55.619363 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.619300    2568 scope.go:117] "RemoveContainer" containerID="6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27"
Jul 28 20:31:55.628920 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.628901    2568 scope.go:117] "RemoveContainer" containerID="6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27"
Jul 28 20:31:55.629195 ip-10-0-128-255 kubenswrapper[2568]: E0728 20:31:55.629175    2568 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27\": container with ID starting with 6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27 not found: ID does not exist" containerID="6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27"
Jul 28 20:31:55.629276 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.629201    2568 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27"} err="failed to get container status \"6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27\": rpc error: code = NotFound desc = could not find container \"6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27\": container with ID starting with 6f021127aafc5be2197bc5640f1a7ff9b3a7209deb1bf2551dd095867e6cca27 not found: ID does not exist"
Jul 28 20:31:55.643747 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.643720    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"]
Jul 28 20:31:55.653586 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.653557    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-60efb4-755dd8787b-xrxjt"]
Jul 28 20:31:55.677284 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.677254    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="af859f3f-6b25-40f0-9d9c-f68729cf342c" path="/var/lib/kubelet/pods/af859f3f-6b25-40f0-9d9c-f68729cf342c/volumes"
Jul 28 20:31:55.677711 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.677691    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c" path="/var/lib/kubelet/pods/e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c/volumes"
Jul 28 20:31:55.694695 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.694666    2568 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"]
Jul 28 20:31:55.709568 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.709543    2568 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-60efb4-76c98d7b9d-4ldzc"]
Jul 28 20:31:55.805949 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:55.805912    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:31:57.676274 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:31:57.676230    2568 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7233b870-bc66-4d2d-9081-a4c6436f0b99" path="/var/lib/kubelet/pods/7233b870-bc66-4d2d-9081-a4c6436f0b99/volumes"
Jul 28 20:32:02.117668 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:02.117629    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:02.206830 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:02.206793    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:02.438869 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:02.438837    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:04.757048 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:04.757011    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:08.659683 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:08.659646    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-l8vf8_1089c743-484f-4e6f-b82b-2cf6249a9b43/manager/0.log"
Jul 28 20:32:08.795214 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:08.795183    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mwzqd_212d3cb5-ad7a-463a-9ab3-7bf10f94afaa/manager/0.log"
Jul 28 20:32:09.339164 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:09.339129    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f4d69945-5694p_9d4e5600-2f7a-43e4-8288-f2337cd1a875/copy-manifests/0.log"
Jul 28 20:32:09.347538 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:09.347513    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f4d69945-5694p_9d4e5600-2f7a-43e4-8288-f2337cd1a875/manager/0.log"
Jul 28 20:32:10.174240 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:10.174210    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754480-hjh42_b99094c3-2863-45bb-8319-ddfdf1c2e5dd/cleanup/0.log"
Jul 28 20:32:10.302261 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:10.302230    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29754495-x4cng_953fcf3e-8065-4d9f-85c0-4e4b521ec81f/cleanup/0.log"
Jul 28 20:32:12.363903 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:12.363871    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-p2kx2_613c075a-25d1-4ba6-a35c-bceafb4a65cf/kuadrant-console-plugin/0.log"
Jul 28 20:32:14.244102 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:14.244067    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d9bfc7fdf-nmlwq_e9190e9f-9c61-4c82-aa8a-b14cc3e1c348/router/0.log"
Jul 28 20:32:14.745714 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:14.745684    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-c96zb_6dc95503-ef27-42fd-a142-0a5122bcfcaf/main/0.log"
Jul 28 20:32:14.983388 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:14.983345    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fjpksb_012824d0-5672-4ba2-858d-51b42153f0c5/main/0.log"
Jul 28 20:32:15.109921 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:15.109831    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-5p4wc_a0f82583-c9b7-4004-8a08-1ff05f5e7b0f/main/0.log"
Jul 28 20:32:15.227958 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:15.227924    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-8hnbs_fdde63f6-f71c-4147-a8f4-eb86c9082924/main/0.log"
Jul 28 20:32:21.286976 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:21.286934    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:21.376201 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:21.376169    2568 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log" limit=1024
Jul 28 20:32:24.045934 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:24.045900    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6w927_66ebf5f5-ab7c-4cb7-9857-b376b202edae/global-pull-secret-syncer/0.log"
Jul 28 20:32:24.177534 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:24.177494    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-547pv_e271958f-3e62-4943-ac71-31f18a0d7369/konnectivity-agent/0.log"
Jul 28 20:32:24.316275 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:24.316191    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-255.ec2.internal_c45489797f52552f43a8393e08ba8c3a/haproxy/0.log"
Jul 28 20:32:29.627596 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:29.627566    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-p2kx2_613c075a-25d1-4ba6-a35c-bceafb4a65cf/kuadrant-console-plugin/0.log"
Jul 28 20:32:31.891340 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:31.891301    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-548b489cb8-kgnw9_0a1ace64-f78c-4dc0-ad9e-fdaa8ff64ad3/cluster-monitoring-operator/0.log"
Jul 28 20:32:32.063791 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:32.063759    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8hccr_3dbc7762-8ad9-4a4a-b513-1a3aba7f845c/node-exporter/0.log"
Jul 28 20:32:32.089595 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:32.089571    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8hccr_3dbc7762-8ad9-4a4a-b513-1a3aba7f845c/kube-rbac-proxy/0.log"
Jul 28 20:32:32.111237 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:32.111214    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-8hccr_3dbc7762-8ad9-4a4a-b513-1a3aba7f845c/init-textfile/0.log"
Jul 28 20:32:33.494373 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494063    2568 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"]
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494562    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c" containerName="revoke-keys"
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494576    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c" containerName="revoke-keys"
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494596    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="af859f3f-6b25-40f0-9d9c-f68729cf342c" containerName="maas-api"
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494602    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="af859f3f-6b25-40f0-9d9c-f68729cf342c" containerName="maas-api"
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494614    2568 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7233b870-bc66-4d2d-9081-a4c6436f0b99" containerName="payload-processing"
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494620    2568 state_mem.go:107] "Deleted CPUSet assignment" podUID="7233b870-bc66-4d2d-9081-a4c6436f0b99" containerName="payload-processing"
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494675    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="7233b870-bc66-4d2d-9081-a4c6436f0b99" containerName="payload-processing"
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494682    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="af859f3f-6b25-40f0-9d9c-f68729cf342c" containerName="maas-api"
Jul 28 20:32:33.494771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.494691    2568 memory_manager.go:368] "RemoveStaleState removing state" podUID="e658b90d-27d9-42b5-8bcd-6fa6f7e88d5c" containerName="revoke-keys"
Jul 28 20:32:33.497803 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.497788    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.500458 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.500431    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z498q\"/\"kube-root-ca.crt\""
Jul 28 20:32:33.500571 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.500495    2568 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-z498q\"/\"openshift-service-ca.crt\""
Jul 28 20:32:33.501106 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.501088    2568 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-z498q\"/\"default-dockercfg-xg8mv\""
Jul 28 20:32:33.509309 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.509288    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"]
Jul 28 20:32:33.559213 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.559179    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-sys\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.559213 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.559215    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-proc\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.559433 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.559244    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxfn2\" (UniqueName: \"kubernetes.io/projected/3bfd93c8-3b9c-413b-9612-ba9d95022569-kube-api-access-jxfn2\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.559433 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.559345    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-lib-modules\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.559433 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.559402    2568 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-podres\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.660672 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.660636    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-podres\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.660870 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.660707    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-sys\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.660870 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.660769    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-sys\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.660870 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.660814    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-proc\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.660870 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.660838    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-podres\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.660870 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.660851    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxfn2\" (UniqueName: \"kubernetes.io/projected/3bfd93c8-3b9c-413b-9612-ba9d95022569-kube-api-access-jxfn2\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.661103 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.660899    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-proc\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.661103 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.660915    2568 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-lib-modules\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.661103 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.661085    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/3bfd93c8-3b9c-413b-9612-ba9d95022569-lib-modules\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.670498 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.670467    2568 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxfn2\" (UniqueName: \"kubernetes.io/projected/3bfd93c8-3b9c-413b-9612-ba9d95022569-kube-api-access-jxfn2\") pod \"perf-node-gather-daemonset-7qm4m\" (UID: \"3bfd93c8-3b9c-413b-9612-ba9d95022569\") " pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.808156 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.808064    2568 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:33.937771 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:33.937740    2568 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"]
Jul 28 20:32:33.940026 ip-10-0-128-255 kubenswrapper[2568]: W0728 20:32:33.939994    2568 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod3bfd93c8_3b9c_413b_9612_ba9d95022569.slice/crio-8786d439b5c75f6f46f8b5ef8ec377ec19330fbb49b8a3640010b07128fda9fb WatchSource:0}: Error finding container 8786d439b5c75f6f46f8b5ef8ec377ec19330fbb49b8a3640010b07128fda9fb: Status 404 returned error can't find the container with id 8786d439b5c75f6f46f8b5ef8ec377ec19330fbb49b8a3640010b07128fda9fb
Jul 28 20:32:34.075300 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:34.075205    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-74c9f884b7-lw2hv_e5087f7a-f48f-4dc9-a7ac-d1a18643367b/networking-console-plugin/0.log"
Jul 28 20:32:34.655954 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:34.655911    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/1.log"
Jul 28 20:32:34.667085 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:34.667062    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-584dcb6b54-9zrns_52f051cf-a638-429d-b7e1-3091504ad33e/console-operator/2.log"
Jul 28 20:32:34.797295 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:34.797256    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m" event={"ID":"3bfd93c8-3b9c-413b-9612-ba9d95022569","Type":"ContainerStarted","Data":"5dfc6415598d4e6ebde97b430e31aa6e777e4bb0af9ed4f56c1494583ad89684"}
Jul 28 20:32:34.797485 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:34.797340    2568 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m" event={"ID":"3bfd93c8-3b9c-413b-9612-ba9d95022569","Type":"ContainerStarted","Data":"8786d439b5c75f6f46f8b5ef8ec377ec19330fbb49b8a3640010b07128fda9fb"}
Jul 28 20:32:34.797485 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:34.797368    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:34.816591 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:34.816545    2568 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m" podStartSLOduration=1.816529896 podStartE2EDuration="1.816529896s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-28 20:32:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-28 20:32:34.814543745 +0000 UTC m=+3467.626105607" watchObservedRunningTime="2026-07-28 20:32:34.816529896 +0000 UTC m=+3467.628091756"
Jul 28 20:32:35.157273 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:35.157240    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-c8dfc9b74-pjltb_e50233f5-80d6-4577-91d3-b1e8b3f5189e/console/0.log"
Jul 28 20:32:35.201431 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:35.201403    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-xvlm7_399fc0cd-afef-48e4-9468-05e6f5037042/download-server/0.log"
Jul 28 20:32:35.742517 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:35.742480    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-549c6d54cf-k96hr_e49ccaaa-5113-4785-82ab-5414bea2f8f6/volume-data-source-validator/0.log"
Jul 28 20:32:36.763587 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:36.763556    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gzdh7_9ce1caf2-c10b-4415-9058-7688ea24cafd/dns/0.log"
Jul 28 20:32:36.784494 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:36.784471    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gzdh7_9ce1caf2-c10b-4415-9058-7688ea24cafd/kube-rbac-proxy/0.log"
Jul 28 20:32:36.867959 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:36.867932    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-jfqbz_605d46ac-45a6-48a9-a859-11a690c22efc/dns-node-resolver/0.log"
Jul 28 20:32:37.533209 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:37.533173    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mmn7j_40db1398-97f2-4b10-b6b0-debf5bdb6c5d/node-ca/0.log"
Jul 28 20:32:38.990688 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:38.990607    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d9bfc7fdf-nmlwq_e9190e9f-9c61-4c82-aa8a-b14cc3e1c348/router/0.log"
Jul 28 20:32:39.853809 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:39.853777    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-wkdkt_ac898250-c4c6-4362-b0c6-f3149a5f299f/serve-healthcheck-canary/0.log"
Jul 28 20:32:40.336918 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:40.336868    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-6qp7r_95d94912-66ec-4945-a6ba-4c551736c24e/insights-operator/0.log"
Jul 28 20:32:40.338867 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:40.338842    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-5885fdb759-6qp7r_95d94912-66ec-4945-a6ba-4c551736c24e/insights-operator/1.log"
Jul 28 20:32:40.428255 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:40.428225    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mqc6s_a5224743-992f-4d1c-86fe-750213fb90f4/kube-rbac-proxy/0.log"
Jul 28 20:32:40.447931 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:40.447906    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mqc6s_a5224743-992f-4d1c-86fe-750213fb90f4/exporter/0.log"
Jul 28 20:32:40.472094 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:40.472072    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-mqc6s_a5224743-992f-4d1c-86fe-750213fb90f4/extractor/0.log"
Jul 28 20:32:40.812018 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:40.811989    2568 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-z498q/perf-node-gather-daemonset-7qm4m"
Jul 28 20:32:43.283824 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:43.283784    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-l8vf8_1089c743-484f-4e6f-b82b-2cf6249a9b43/manager/0.log"
Jul 28 20:32:43.692041 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:43.692002    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-mwzqd_212d3cb5-ad7a-463a-9ab3-7bf10f94afaa/manager/0.log"
Jul 28 20:32:44.123984 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:44.123941    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f4d69945-5694p_9d4e5600-2f7a-43e4-8288-f2337cd1a875/manager/0.log"
Jul 28 20:32:44.148641 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:44.148613    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5f4d69945-5694p_9d4e5600-2f7a-43e4-8288-f2337cd1a875/copy-manifests/0.log"
Jul 28 20:32:44.539243 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:44.539207    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d9bfc7fdf-nmlwq_e9190e9f-9c61-4c82-aa8a-b14cc3e1c348/router/0.log"
Jul 28 20:32:45.719031 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:45.718997    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-69b7dc5fd4-w9sn8_340d826d-108e-41f7-96ab-c3facd568481/manager/0.log"
Jul 28 20:32:46.501884 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:46.501856    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-5d9bfc7fdf-nmlwq_e9190e9f-9c61-4c82-aa8a-b14cc3e1c348/router/0.log"
Jul 28 20:32:50.516107 ip-10-0-128-255 kubenswrapper[2568]: I0728 20:32:50.516071    2568 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-cscvw_2bd54e8e-d9e9-469d-bb92-8c465c918ec5/migrator/0.log"