Jul 25 01:09:56.572708 ip-10-0-129-103 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Jul 25 01:09:56.572721 ip-10-0-129-103 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Jul 25 01:09:56.572730 ip-10-0-129-103 systemd[1]: kubelet.service: Failed with result 'resources'.
Jul 25 01:09:56.573034 ip-10-0-129-103 systemd[1]: Failed to start Kubernetes Kubelet.
Jul 25 01:10:06.810988 ip-10-0-129-103 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Jul 25 01:10:06.811004 ip-10-0-129-103 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 91e1c939b4424cb6a8f816baec97e9fb --
Jul 25 01:12:00.300718 ip-10-0-129-103 systemd[1]: Starting Kubernetes Kubelet...
Jul 25 01:12:00.661887 ip-10-0-129-103 kubenswrapper[2562]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 01:12:00.661887 ip-10-0-129-103 kubenswrapper[2562]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Jul 25 01:12:00.661887 ip-10-0-129-103 kubenswrapper[2562]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 01:12:00.661887 ip-10-0-129-103 kubenswrapper[2562]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Jul 25 01:12:00.661887 ip-10-0-129-103 kubenswrapper[2562]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Jul 25 01:12:00.663589 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.663499    2562 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Jul 25 01:12:00.668165 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668144    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:12:00.668165 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668161    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:12:00.668165 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668165    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:12:00.668165 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668169    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668174    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668179    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668184    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668189    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668193    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668198    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668202    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668205    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668208    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668210    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668213    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668215    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668218    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668221    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668224    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668226    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668228    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668232    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:12:00.668318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668236    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668240    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668246    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668248    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668252    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668255    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668258    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668261    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668264    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668266    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668274    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668277    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668280    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668282    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668285    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668289    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668291    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668294    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668297    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:12:00.668804 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668299    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668302    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668304    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668307    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668310    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668312    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668315    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668318    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668320    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668323    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668327    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668329    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668332    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668335    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668338    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668341    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668343    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668346    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668348    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:12:00.669271 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668351    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668355    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668359    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668362    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668365    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668367    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668370    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668372    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668375    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668379    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668382    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668384    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668387    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668390    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668392    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668395    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668397    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668400    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668402    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668405    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:12:00.669751 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668407    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668409    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668412    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668415    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668419    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668421    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668424    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668426    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668428    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668447    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668450    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668452    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668455    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668458    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668461    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668463    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668466    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668469    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.668471    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:12:00.670230 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669155    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669161    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669165    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669169    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669173    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669176    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669180    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669183    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669186    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669189    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669192    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669194    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669197    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669200    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669202    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669205    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669207    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669210    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669213    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:12:00.670701 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669215    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669218    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669221    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669224    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669226    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669229    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669232    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669234    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669237    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669239    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669242    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669244    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669247    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669249    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669252    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669256    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669259    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669261    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669264    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:12:00.671179 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669266    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669269    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669271    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669274    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669277    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669279    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669282    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669285    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669288    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669290    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669293    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669295    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669299    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669301    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669304    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669306    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669309    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669312    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669314    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669317    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:12:00.671665 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669320    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669322    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669324    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669327    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669330    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669332    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669335    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669338    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669341    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669343    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669346    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669349    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669351    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669354    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669356    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669359    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669361    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669364    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669366    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669368    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:12:00.672141 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669371    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669373    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669376    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669381    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669384    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669388    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669392    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669395    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669398    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669401    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669404    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669406    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669409    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669411    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669414    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669416    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669419    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669421    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669424    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:12:00.672667 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669426    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669443    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669850    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669855    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669859    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669863    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669867    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669869    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669872    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669875    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669878    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669880    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669883    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669885    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669888    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669890    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669893    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669895    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669902    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:12:00.673124 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669904    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669907    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669912    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669914    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669917    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669919    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669922    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669925    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669927    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669930    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669933    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669935    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669938    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669940    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669943    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669946    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669948    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669951    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669954    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669956    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:12:00.673607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669959    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669961    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669964    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669966    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669969    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669973    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669977    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669981    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669984    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669986    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669990    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669993    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669996    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.669998    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670003    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670006    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670008    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670011    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670014    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670016    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:12:00.674116 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670019    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670021    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670024    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670026    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670029    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670032    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670034    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670037    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670039    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670042    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670046    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670048    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670051    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670053    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670056    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670059    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670061    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670063    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670066    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670069    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:12:00.674612 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670071    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670074    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670076    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670079    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670081    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670084    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670088    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670091    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670094    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670096    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670099    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670102    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670106    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670109    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670112    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670115    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670121    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670124    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670127    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670129    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:12:00.675181 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670132    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670135    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670199    2562 flags.go:64] FLAG: --address="0.0.0.0"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670211    2562 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670217    2562 flags.go:64] FLAG: --anonymous-auth="true"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670221    2562 flags.go:64] FLAG: --application-metrics-count-limit="100"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670224    2562 flags.go:64] FLAG: --authentication-token-webhook="false"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670227    2562 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670232    2562 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670238    2562 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670242    2562 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670245    2562 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670248    2562 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670251    2562 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670254    2562 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670257    2562 flags.go:64] FLAG: --cgroup-root=""
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670260    2562 flags.go:64] FLAG: --cgroups-per-qos="true"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670263    2562 flags.go:64] FLAG: --client-ca-file=""
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670268    2562 flags.go:64] FLAG: --cloud-provider="external"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670271    2562 flags.go:64] FLAG: --cluster-dns="[]"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670275    2562 flags.go:64] FLAG: --cluster-domain=""
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670278    2562 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670281    2562 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670284    2562 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Jul 25 01:12:00.675718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670287    2562 flags.go:64] FLAG: --container-log-max-files="5"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670291    2562 flags.go:64] FLAG: --container-log-max-size="10Mi"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670294    2562 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670297    2562 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670301    2562 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670304    2562 flags.go:64] FLAG: --contention-profiling="false"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670307    2562 flags.go:64] FLAG: --cpu-cfs-quota="true"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670311    2562 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670314    2562 flags.go:64] FLAG: --cpu-manager-policy="none"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670317    2562 flags.go:64] FLAG: --cpu-manager-policy-options=""
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670320    2562 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670323    2562 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670326    2562 flags.go:64] FLAG: --enable-debugging-handlers="true"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670329    2562 flags.go:64] FLAG: --enable-load-reader="false"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670332    2562 flags.go:64] FLAG: --enable-server="true"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670335    2562 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670339    2562 flags.go:64] FLAG: --event-burst="100"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670342    2562 flags.go:64] FLAG: --event-qps="50"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670354    2562 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670358    2562 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670360    2562 flags.go:64] FLAG: --eviction-hard=""
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670364    2562 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670367    2562 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670370    2562 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670373    2562 flags.go:64] FLAG: --eviction-soft=""
Jul 25 01:12:00.676296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670376    2562 flags.go:64] FLAG: --eviction-soft-grace-period=""
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670380    2562 flags.go:64] FLAG: --exit-on-lock-contention="false"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670383    2562 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670387    2562 flags.go:64] FLAG: --experimental-mounter-path=""
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670390    2562 flags.go:64] FLAG: --fail-cgroupv1="false"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670393    2562 flags.go:64] FLAG: --fail-swap-on="true"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670395    2562 flags.go:64] FLAG: --feature-gates=""
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670399    2562 flags.go:64] FLAG: --file-check-frequency="20s"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670402    2562 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670405    2562 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670408    2562 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670412    2562 flags.go:64] FLAG: --healthz-port="10248"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670415    2562 flags.go:64] FLAG: --help="false"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670418    2562 flags.go:64] FLAG: --hostname-override="ip-10-0-129-103.ec2.internal"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670421    2562 flags.go:64] FLAG: --housekeeping-interval="10s"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670424    2562 flags.go:64] FLAG: --http-check-frequency="20s"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670427    2562 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670442    2562 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670446    2562 flags.go:64] FLAG: --image-gc-high-threshold="85"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670449    2562 flags.go:64] FLAG: --image-gc-low-threshold="80"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670452    2562 flags.go:64] FLAG: --image-service-endpoint=""
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670455    2562 flags.go:64] FLAG: --kernel-memcg-notification="false"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670458    2562 flags.go:64] FLAG: --kube-api-burst="100"
Jul 25 01:12:00.677332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670461    2562 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670464    2562 flags.go:64] FLAG: --kube-api-qps="50"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670467    2562 flags.go:64] FLAG: --kube-reserved=""
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670470    2562 flags.go:64] FLAG: --kube-reserved-cgroup=""
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670473    2562 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670475    2562 flags.go:64] FLAG: --kubelet-cgroups=""
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670478    2562 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670481    2562 flags.go:64] FLAG: --lock-file=""
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670484    2562 flags.go:64] FLAG: --log-cadvisor-usage="false"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670487    2562 flags.go:64] FLAG: --log-flush-frequency="5s"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670491    2562 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670496    2562 flags.go:64] FLAG: --log-json-split-stream="false"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670500    2562 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670503    2562 flags.go:64] FLAG: --log-text-split-stream="false"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670505    2562 flags.go:64] FLAG: --logging-format="text"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670508    2562 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670512    2562 flags.go:64] FLAG: --make-iptables-util-chains="true"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670514    2562 flags.go:64] FLAG: --manifest-url=""
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670517    2562 flags.go:64] FLAG: --manifest-url-header=""
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670521    2562 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670525    2562 flags.go:64] FLAG: --max-open-files="1000000"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670529    2562 flags.go:64] FLAG: --max-pods="110"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670532    2562 flags.go:64] FLAG: --maximum-dead-containers="-1"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670535    2562 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670538    2562 flags.go:64] FLAG: --memory-manager-policy="None"
Jul 25 01:12:00.678368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670541    2562 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670543    2562 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670546    2562 flags.go:64] FLAG: --node-ip="0.0.0.0"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670549    2562 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670556    2562 flags.go:64] FLAG: --node-status-max-images="50"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670559    2562 flags.go:64] FLAG: --node-status-update-frequency="10s"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670562    2562 flags.go:64] FLAG: --oom-score-adj="-999"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670565    2562 flags.go:64] FLAG: --pod-cidr=""
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670567    2562 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670572    2562 flags.go:64] FLAG: --pod-manifest-path=""
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670575    2562 flags.go:64] FLAG: --pod-max-pids="-1"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670578    2562 flags.go:64] FLAG: --pods-per-core="0"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670581    2562 flags.go:64] FLAG: --port="10250"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670584    2562 flags.go:64] FLAG: --protect-kernel-defaults="false"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670587    2562 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-04be05a7cb0c9d30a"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670590    2562 flags.go:64] FLAG: --qos-reserved=""
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670593    2562 flags.go:64] FLAG: --read-only-port="10255"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670596    2562 flags.go:64] FLAG: --register-node="true"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670600    2562 flags.go:64] FLAG: --register-with-taints=""
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670603    2562 flags.go:64] FLAG: --registry-burst="10"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670607    2562 flags.go:64] FLAG: --registry-qps="5"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670609    2562 flags.go:64] FLAG: --reserved-cpus=""
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670612    2562 flags.go:64] FLAG: --reserved-memory=""
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670616    2562 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670619    2562 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Jul 25 01:12:00.679501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670621    2562 flags.go:64] FLAG: --rotate-certificates="false"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670624    2562 flags.go:64] FLAG: --rotate-server-certificates="false"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670628    2562 flags.go:64] FLAG: --runonce="false"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670631    2562 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670634    2562 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670637    2562 flags.go:64] FLAG: --seccomp-default="false"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670641    2562 flags.go:64] FLAG: --serialize-image-pulls="true"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670643    2562 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670646    2562 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670649    2562 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670652    2562 flags.go:64] FLAG: --storage-driver-password="root"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670655    2562 flags.go:64] FLAG: --storage-driver-secure="false"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670658    2562 flags.go:64] FLAG: --storage-driver-table="stats"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670661    2562 flags.go:64] FLAG: --storage-driver-user="root"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670663    2562 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670666    2562 flags.go:64] FLAG: --sync-frequency="1m0s"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670669    2562 flags.go:64] FLAG: --system-cgroups=""
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670672    2562 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670677    2562 flags.go:64] FLAG: --system-reserved-cgroup=""
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670679    2562 flags.go:64] FLAG: --tls-cert-file=""
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670682    2562 flags.go:64] FLAG: --tls-cipher-suites="[]"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670687    2562 flags.go:64] FLAG: --tls-min-version=""
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670689    2562 flags.go:64] FLAG: --tls-private-key-file=""
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670692    2562 flags.go:64] FLAG: --topology-manager-policy="none"
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670695    2562 flags.go:64] FLAG: --topology-manager-policy-options=""
Jul 25 01:12:00.680558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670699    2562 flags.go:64] FLAG: --topology-manager-scope="container"
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670702    2562 flags.go:64] FLAG: --v="2"
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670706    2562 flags.go:64] FLAG: --version="false"
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670711    2562 flags.go:64] FLAG: --vmodule=""
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670715    2562 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.670718    2562 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670821    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670824    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670827    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670831    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670834    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670837    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670839    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670842    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670844    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670847    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670849    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670852    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670854    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670857    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670859    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:12:00.681455 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670862    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670865    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670868    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670870    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670873    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670875    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670878    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670881    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670883    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670885    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670888    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670892    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670895    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670897    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670900    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670903    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670905    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670908    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670911    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:12:00.682318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670913    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670916    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670919    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670922    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670925    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670929    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670931    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670934    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670937    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670940    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670943    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670945    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670948    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670950    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670953    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670956    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670958    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670961    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670963    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:12:00.682929 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670966    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670968    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670972    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670975    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670978    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670983    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670986    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670989    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670992    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670995    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.670998    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671000    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671003    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671013    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671017    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671020    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671023    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671025    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671028    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671030    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:12:00.683554 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671033    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671036    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671038    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671041    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671044    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671047    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671049    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671052    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671055    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671057    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671060    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671062    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671066    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671070    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671072    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671075    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671078    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671082    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671086    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671088    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:12:00.684039 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671091    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671094    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671096    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671100    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671102    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.671105    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.671550    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.678907    2562 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.678927    2562 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.678952    2562 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.678962    2562 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679052    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679060    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679065    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679071    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:12:00.684607 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679076    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679080    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679085    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679089    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679094    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679099    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679104    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679108    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679113    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679117    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679121    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679126    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679130    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679134    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679138    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679143    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679147    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679151    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679155    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679160    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:12:00.685057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679164    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679168    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679172    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679176    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679181    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679187    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679191    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679196    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679200    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679205    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679209    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679213    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679217    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679221    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679225    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679229    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679234    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679238    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679242    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:12:00.685586 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679246    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679250    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679254    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679258    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679262    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679266    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679271    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679275    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679279    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679283    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679287    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679292    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679298    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679303    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679308    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679313    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679318    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679322    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679327    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679333    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:12:00.686315 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679338    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679342    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679346    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679350    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679354    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679359    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679363    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679367    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679371    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679375    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679380    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679384    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679388    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679393    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679397    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679401    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679405    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679409    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679414    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679418    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:12:00.686899 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679422    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679427    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679449    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679454    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679458    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679464    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679470    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679475    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679480    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679485    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679490    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679494    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679506    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679510    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679516    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:12:00.687445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679522    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.679530    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679730    2562 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679738    2562 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679743    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679747    2562 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679752    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679756    2562 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679761    2562 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679765    2562 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679769    2562 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679773    2562 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679777    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679782    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Jul 25 01:12:00.687845 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679786    2562 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679791    2562 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679795    2562 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679800    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679804    2562 feature_gate.go:328] unrecognized feature gate: OVNObservability
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679809    2562 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679813    2562 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679817    2562 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679821    2562 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679825    2562 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679830    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679834    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679838    2562 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679842    2562 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679847    2562 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679851    2562 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679856    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679861    2562 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679865    2562 feature_gate.go:328] unrecognized feature gate: EventTTL
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679869    2562 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Jul 25 01:12:00.688191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679873    2562 feature_gate.go:328] unrecognized feature gate: SignatureStores
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679877    2562 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679882    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679886    2562 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679890    2562 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679895    2562 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679899    2562 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679903    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679907    2562 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679911    2562 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679915    2562 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679920    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679924    2562 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679928    2562 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679932    2562 feature_gate.go:328] unrecognized feature gate: PinnedImages
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679937    2562 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679941    2562 feature_gate.go:328] unrecognized feature gate: Example2
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679946    2562 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679951    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679955    2562 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Jul 25 01:12:00.689106 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679959    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679963    2562 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679968    2562 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679973    2562 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679977    2562 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679981    2562 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679986    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679990    2562 feature_gate.go:328] unrecognized feature gate: Example
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.679995    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680000    2562 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680004    2562 feature_gate.go:328] unrecognized feature gate: NewOLM
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680009    2562 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680013    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680017    2562 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680022    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680029    2562 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680035    2562 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680039    2562 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680044    2562 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680048    2562 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Jul 25 01:12:00.689926 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680053    2562 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680057    2562 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680061    2562 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680065    2562 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680069    2562 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680074    2562 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680078    2562 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680082    2562 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680087    2562 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680091    2562 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680095    2562 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680099    2562 feature_gate.go:328] unrecognized feature gate: OSStreams
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680103    2562 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680108    2562 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680113    2562 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680117    2562 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680121    2562 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680125    2562 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680132    2562 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680137    2562 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Jul 25 01:12:00.690495 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680142    2562 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680147    2562 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680153    2562 feature_gate.go:328] unrecognized feature gate: DualReplica
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680159    2562 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680164    2562 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680169    2562 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:00.680173    2562 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.680181    2562 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.680324    2562 server.go:962] "Client rotation is on, will bootstrap in background"
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.682739    2562 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.684097    2562 server.go:1019] "Starting client certificate rotation"
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.684205    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.684272    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Jul 25 01:12:00.690964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.686332    2562 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 25 01:12:00.692241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.692222    2562 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Jul 25 01:12:00.701609 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.701591    2562 log.go:25] "Validated CRI v1 runtime API"
Jul 25 01:12:00.705699 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.705686    2562 log.go:25] "Validated CRI v1 image API"
Jul 25 01:12:00.707035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.707020    2562 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Jul 25 01:12:00.709962 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.709939    2562 fs.go:135] Filesystem UUIDs: map[44c05367-e058-4bcf-977f-d9020df66476:/dev/nvme0n1p3 77ffd986-770b-4902-b5bb-cd152dfd8517:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2]
Jul 25 01:12:00.710032 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.709961    2562 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Jul 25 01:12:00.712824 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.712799    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 25 01:12:00.714991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.714879    2562 manager.go:216] Machine: {Timestamp:2026-07-25 01:12:00.713664066 +0000 UTC m=+0.311166012 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101764 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2491a68fd8e96ca41f3913b5d5829a SystemUUID:ec2491a6-8fd8-e96c-a41f-3913b5d5829a BootID:91e1c939-b442-4cb6-a8f8-16baec97e9fb Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:b1:bd:eb:54:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:b1:bd:eb:54:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:b9:4d:0d:f9:4b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Jul 25 01:12:00.714991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.714985    2562 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Jul 25 01:12:00.715150 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.715134    2562 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Jul 25 01:12:00.717527 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.717504    2562 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Jul 25 01:12:00.717669 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.717530    2562 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-129-103.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Jul 25 01:12:00.718094 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.718085    2562 topology_manager.go:138] "Creating topology manager with none policy"
Jul 25 01:12:00.718133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.718097    2562 container_manager_linux.go:309] "Creating device plugin manager"
Jul 25 01:12:00.718133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.718104    2562 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 25 01:12:00.718133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.718117    2562 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Jul 25 01:12:00.718888 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.718878    2562 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Jul 25 01:12:00.721536 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.721525    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 25 01:12:00.721639 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.721629    2562 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Jul 25 01:12:00.721677 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.721658    2562 kubelet.go:486] "Attempting to sync node with API server"
Jul 25 01:12:00.721677 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.721667    2562 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Jul 25 01:12:00.721771 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.721681    2562 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Jul 25 01:12:00.721771 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.721691    2562 kubelet.go:398] "Adding apiserver pod source"
Jul 25 01:12:00.721771 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.721699    2562 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Jul 25 01:12:00.722021 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.722010    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 25 01:12:00.722069 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.722027    2562 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Jul 25 01:12:00.723804 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.723782    2562 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1"
Jul 25 01:12:00.725474 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.725459    2562 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Jul 25 01:12:00.725535 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.725493    2562 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Jul 25 01:12:00.726932 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726920    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Jul 25 01:12:00.726977 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726936    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Jul 25 01:12:00.726977 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726942    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Jul 25 01:12:00.726977 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726947    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Jul 25 01:12:00.726977 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726955    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Jul 25 01:12:00.726977 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726960    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Jul 25 01:12:00.726977 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726967    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Jul 25 01:12:00.726977 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726973    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Jul 25 01:12:00.727162 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726987    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Jul 25 01:12:00.727162 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.726993    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Jul 25 01:12:00.727162 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.727001    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Jul 25 01:12:00.727162 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.727014    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Jul 25 01:12:00.727561 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.727549    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Jul 25 01:12:00.727592 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.727563    2562 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Jul 25 01:12:00.729884 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.729866    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Jul 25 01:12:00.730141 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.730014    2562 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-129-103.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Jul 25 01:12:00.731201 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.731181    2562 server.go:1268] "Started kubelet"
Jul 25 01:12:00.731555 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.731530    2562 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-129-103.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Jul 25 01:12:00.731912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.731877    2562 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Jul 25 01:12:00.732118 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.732034    2562 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Jul 25 01:12:00.732186 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.732143    2562 server_v1.go:49] "podresources" method="list" useActivePods=true
Jul 25 01:12:00.732297 ip-10-0-129-103 systemd[1]: Started Kubernetes Kubelet.
Jul 25 01:12:00.733713 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.733697    2562 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Jul 25 01:12:00.734080 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.734068    2562 server.go:310] "Adding debug handlers to kubelet server"
Jul 25 01:12:00.734919 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.734893    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fmwk7"
Jul 25 01:12:00.737860 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.737844    2562 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Jul 25 01:12:00.738681 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.737747    2562 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-129-103.ec2.internal.18c5633849e24653  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-129-103.ec2.internal,UID:ip-10-0-129-103.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-129-103.ec2.internal,},FirstTimestamp:2026-07-25 01:12:00.731154003 +0000 UTC m=+0.328655949,LastTimestamp:2026-07-25 01:12:00.731154003 +0000 UTC m=+0.328655949,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-129-103.ec2.internal,}"
Jul 25 01:12:00.741075 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.741057    2562 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Jul 25 01:12:00.741154 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.741074    2562 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Jul 25 01:12:00.741302 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.741286    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:00.741705 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.741686    2562 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Jul 25 01:12:00.741705 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.741699    2562 volume_manager.go:311] "The desired_state_of_world populator starts"
Jul 25 01:12:00.741857 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.741714    2562 volume_manager.go:313] "Starting Kubelet Volume Manager"
Jul 25 01:12:00.741857 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.741818    2562 reconstruct.go:97] "Volume reconstruction finished"
Jul 25 01:12:00.741857 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.741827    2562 reconciler.go:29] "Reconciler: start to sync state"
Jul 25 01:12:00.742041 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.742026    2562 factory.go:55] Registering systemd factory
Jul 25 01:12:00.742144 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.742044    2562 factory.go:223] Registration of the systemd container factory successfully
Jul 25 01:12:00.742279 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.742263    2562 factory.go:153] Registering CRI-O factory
Jul 25 01:12:00.742279 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.742279    2562 factory.go:223] Registration of the crio container factory successfully
Jul 25 01:12:00.742400 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.742329    2562 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Jul 25 01:12:00.742400 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.742349    2562 factory.go:103] Registering Raw factory
Jul 25 01:12:00.742400 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.742359    2562 manager.go:1196] Started watching for new ooms in manager
Jul 25 01:12:00.742954 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.742937    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-fmwk7"
Jul 25 01:12:00.743136 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.743126    2562 manager.go:349] Starting recovery of all containers
Jul 25 01:12:00.753488 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.753474    2562 manager.go:354] Recovery completed
Jul 25 01:12:00.758010 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.757997    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:12:00.758971 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.758958    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:12:00.760136 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.760115    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:12:00.760218 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.760147    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:12:00.760218 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.760157    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:12:00.761512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.760935    2562 cpu_manager.go:222] "Starting CPU manager" policy="none"
Jul 25 01:12:00.761512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.760948    2562 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Jul 25 01:12:00.761512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.760956    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:12:00.761512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.760967    2562 state_mem.go:36] "Initialized new in-memory state store"
Jul 25 01:12:00.761967 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.761947    2562 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-129-103.ec2.internal\" not found" node="ip-10-0-129-103.ec2.internal"
Jul 25 01:12:00.763079 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.763063    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:12:00.763166 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.763092    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:12:00.763166 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.763105    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:12:00.763804 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.763793    2562 policy_none.go:49] "None policy: Start"
Jul 25 01:12:00.763842 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.763807    2562 memory_manager.go:187] "Starting memorymanager" policy="None"
Jul 25 01:12:00.763842 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.763818    2562 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Jul 25 01:12:00.765631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.765620    2562 policy_none.go:47] "Start"
Jul 25 01:12:00.803138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.802959    2562 manager.go:337] "Starting Device Plugin manager"
Jul 25 01:12:00.803236 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.803158    2562 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Jul 25 01:12:00.803236 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.803173    2562 server.go:87] "Starting device plugin registration server"
Jul 25 01:12:00.803448 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.803423    2562 eviction_manager.go:189] "Eviction manager: starting control loop"
Jul 25 01:12:00.803503 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.803454    2562 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Jul 25 01:12:00.803626 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.803565    2562 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Jul 25 01:12:00.803680 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.803651    2562 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Jul 25 01:12:00.803680 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.803660    2562 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Jul 25 01:12:00.804162 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.804139    2562 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Jul 25 01:12:00.804227 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.804201    2562 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:00.903612 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.903583    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:12:00.905673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.904591    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:12:00.905673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.904625    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:12:00.905673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.904640    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:12:00.905673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.904667    2562 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-129-103.ec2.internal"
Jul 25 01:12:00.914815 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.914771    2562 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-129-103.ec2.internal"
Jul 25 01:12:00.914815 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.914794    2562 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-129-103.ec2.internal\": node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:00.933145 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.933123    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:00.941541 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.941503    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Jul 25 01:12:00.942711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.942695    2562 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Jul 25 01:12:00.942817 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.942736    2562 status_manager.go:244] "Starting to sync pod status with apiserver"
Jul 25 01:12:00.942817 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.942767    2562 kubelet.go:2448] "Starting kubelet main sync loop"
Jul 25 01:12:00.942817 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:00.942808    2562 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Jul 25 01:12:00.945183 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:00.945165    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:12:02.160027 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.159969    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:02.170308 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.170261    2562 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"]
Jul 25 01:12:02.170372 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.170354    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:12:02.171855 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.171835    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:12:02.171958 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.171864    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:12:02.171958 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.171875    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:12:02.174077 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174066    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:12:02.174221 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174206    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.174267 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174232    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:12:02.174854 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174839    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:12:02.174927 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174867    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:12:02.174927 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174881    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:12:02.174927 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174900    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:12:02.174927 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174926    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:12:02.175125 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.174935    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:12:02.177031 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.177016    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.177110 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.177046    2562 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Jul 25 01:12:02.177673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.177659    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientMemory"
Jul 25 01:12:02.177748 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.177685    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasNoDiskPressure"
Jul 25 01:12:02.177748 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.177695    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeHasSufficientPID"
Jul 25 01:12:02.202624 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.202601    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-103.ec2.internal\" not found" node="ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.206859 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.206845    2562 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-129-103.ec2.internal\" not found" node="ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.261126 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.261100    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:02.270476 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.270444    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ed3ed7be1c2695b0c97f92390b9feef-config\") pod \"kube-apiserver-proxy-ip-10-0-129-103.ec2.internal\" (UID: \"1ed3ed7be1c2695b0c97f92390b9feef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.270476 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.270480    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b5e70bad49fe9daa9a739742c8d5fb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal\" (UID: \"2b5e70bad49fe9daa9a739742c8d5fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.270596 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.270499    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b5e70bad49fe9daa9a739742c8d5fb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal\" (UID: \"2b5e70bad49fe9daa9a739742c8d5fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.361944 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.361882    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:02.371279 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.371259    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ed3ed7be1c2695b0c97f92390b9feef-config\") pod \"kube-apiserver-proxy-ip-10-0-129-103.ec2.internal\" (UID: \"1ed3ed7be1c2695b0c97f92390b9feef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.371331 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.371285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b5e70bad49fe9daa9a739742c8d5fb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal\" (UID: \"2b5e70bad49fe9daa9a739742c8d5fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.371331 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.371305    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b5e70bad49fe9daa9a739742c8d5fb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal\" (UID: \"2b5e70bad49fe9daa9a739742c8d5fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.371405 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.371346    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2b5e70bad49fe9daa9a739742c8d5fb2-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal\" (UID: \"2b5e70bad49fe9daa9a739742c8d5fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.371405 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.371373    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/1ed3ed7be1c2695b0c97f92390b9feef-config\") pod \"kube-apiserver-proxy-ip-10-0-129-103.ec2.internal\" (UID: \"1ed3ed7be1c2695b0c97f92390b9feef\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.371405 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.371375    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/2b5e70bad49fe9daa9a739742c8d5fb2-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal\" (UID: \"2b5e70bad49fe9daa9a739742c8d5fb2\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.462449 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.462421    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:02.504905 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.504874    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.509774 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.509613    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.562760 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.562732    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:02.663225 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.663199    2562 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-129-103.ec2.internal\" not found"
Jul 25 01:12:02.710164 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.710137    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:12:02.768354 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.768333    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.778297 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.778275    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 25 01:12:02.779677 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.779662    2562 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal"
Jul 25 01:12:02.789948 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.789928    2562 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Jul 25 01:12:02.810108 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.810085    2562 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Jul 25 01:12:02.810234 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.810215    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 25 01:12:02.810290 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.810247    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 25 01:12:02.810290 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.810247    2562 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Jul 25 01:12:02.849418 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.849386    2562 apiserver.go:52] "Watching apiserver"
Jul 25 01:12:02.864869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.864838    2562 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Jul 25 01:12:02.865187 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.865167    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal","openshift-cluster-node-tuning-operator/tuned-fcp6w","openshift-image-registry/node-ca-szrr5","openshift-multus/multus-additional-cni-plugins-vl7nz","openshift-network-diagnostics/network-check-target-5fqmf","openshift-network-operator/iptables-alerter-fjcfh","openshift-ovn-kubernetes/ovnkube-node-s7gf8","kube-system/global-pull-secret-syncer-c46qr","kube-system/konnectivity-agent-8qxdx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw","openshift-dns/node-resolver-hljng","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal","openshift-multus/multus-wwkc9","openshift-multus/network-metrics-daemon-w6nf6"]
Jul 25 01:12:02.867744 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.867703    2562 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Jul 25 01:12:02.868498 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.868480    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.870563 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.870549    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.871210 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.871195    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ds68h\""
Jul 25 01:12:02.871279 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.871244    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Jul 25 01:12:02.871279 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.871266    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:12:02.872741 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.872729    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.872921 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.872905    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Jul 25 01:12:02.872921 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.872908    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-l5jpd\""
Jul 25 01:12:02.873131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.873117    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Jul 25 01:12:02.873225 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.873212    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Jul 25 01:12:02.874443 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874397    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-24 01:07:01 +0000 UTC" deadline="2027-12-28 04:52:32.681374229 +0000 UTC"
Jul 25 01:12:02.874443 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874441    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12507h40m29.806935251s"
Jul 25 01:12:02.874579 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874460    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2x96\" (UniqueName: \"kubernetes.io/projected/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-kube-api-access-q2x96\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.874579 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874487    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysconfig\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874579 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874503    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-kubernetes\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874579 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874520    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-tuned\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874579 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874564    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-host\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874586    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-serviceca\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874606    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-modprobe-d\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874628    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-systemd\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874656    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-host\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vgbq\" (UniqueName: \"kubernetes.io/projected/5ffc13fa-3522-4ec9-8d85-6681936e8104-kube-api-access-2vgbq\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874724    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysctl-d\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874757    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-lib-modules\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874764    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874782    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ffc13fa-3522-4ec9-8d85-6681936e8104-tmp\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.874807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysctl-conf\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.875259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874833    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-run\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.875259 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.874836    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:02.875259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874850    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-sys\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.875259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874867    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-var-lib-kubelet\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.875259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874945    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bgpx6\""
Jul 25 01:12:02.875259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.874974    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Jul 25 01:12:02.875259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.875261    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Jul 25 01:12:02.875515 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.875369    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Jul 25 01:12:02.875515 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.875372    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Jul 25 01:12:02.875515 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.875450    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Jul 25 01:12:02.876901 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.876875    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:02.879052 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.879036    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-fbkn2\""
Jul 25 01:12:02.879185 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.879123    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.879316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.879301    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Jul 25 01:12:02.879413 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.879377    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Jul 25 01:12:02.879492 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.879414    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Jul 25 01:12:02.880359 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.880344    2562 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Jul 25 01:12:02.881367 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.881326    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:02.881462 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.881380    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:02.881462 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.881459    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Jul 25 01:12:02.881574 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.881504    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-n9k86\""
Jul 25 01:12:02.881574 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.881535    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Jul 25 01:12:02.881732 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.881718    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Jul 25 01:12:02.881781 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.881768    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Jul 25 01:12:02.882967 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.882948    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Jul 25 01:12:02.883082 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.882949    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Jul 25 01:12:02.884029 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.884015    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:02.886117 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.886099    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xqwf7\""
Jul 25 01:12:02.886233 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.886220    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:02.886341 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.886327    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Jul 25 01:12:02.886614 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.886597    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Jul 25 01:12:02.888578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.888557    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:02.889007 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.888972    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Jul 25 01:12:02.889106 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.889027    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-jwrfm\""
Jul 25 01:12:02.891332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.889793    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Jul 25 01:12:02.891332 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.890079    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Jul 25 01:12:02.891583 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.891563    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Jul 25 01:12:02.891676 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.891662    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.891864 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.891760    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Jul 25 01:12:02.891951 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.891880    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bnqsg\""
Jul 25 01:12:02.893993 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.893967    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Jul 25 01:12:02.894609 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.894594    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-pjstm\""
Jul 25 01:12:02.894882 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.894867    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:02.894934 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:02.894922    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:02.906946 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.906926    2562 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q6n4q"
Jul 25 01:12:02.914152 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.914100    2562 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q6n4q"
Jul 25 01:12:02.969222 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.969204    2562 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Jul 25 01:12:02.975075 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975056    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-modprobe-d\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975160 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-systemd\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975160 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975098    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vgbq\" (UniqueName: \"kubernetes.io/projected/5ffc13fa-3522-4ec9-8d85-6681936e8104-kube-api-access-2vgbq\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975160 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975116    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-cni-netd\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.975160 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-daemon-config\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.975347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-hosts-file\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:02.975347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975196    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhvsh\" (UniqueName: \"kubernetes.io/projected/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-kube-api-access-qhvsh\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:02.975347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975195    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-systemd\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975222    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysctl-d\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975229    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-modprobe-d\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ffc13fa-3522-4ec9-8d85-6681936e8104-tmp\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975339    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-cni-bin\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975338    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysctl-d\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975391    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975437    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-k8s-cni-cncf-io\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975478    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-kubelet\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975502    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cnibin\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975530    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-ovn\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975541    2562 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975551    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8237157-cb64-498e-8759-7715d46af050-agent-certs\") pod \"konnectivity-agent-8qxdx\" (UID: \"b8237157-cb64-498e-8759-7715d46af050\") " pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:02.975591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975585    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-multus-certs\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975605    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8237157-cb64-498e-8759-7715d46af050-konnectivity-ca\") pod \"konnectivity-agent-8qxdx\" (UID: \"b8237157-cb64-498e-8759-7715d46af050\") " pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975622    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-kubernetes\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975636    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-host\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975652    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ed77e69-f8d9-4541-bcbd-af44efbcd177-iptables-alerter-script\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975675    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-kubelet\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975681    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-kubernetes\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-systemd\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-host\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975731    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975752    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-host\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975766    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-registration-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975781    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-cni-multus\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975820    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-host\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975825    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-system-cni-dir\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975862    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ed77e69-f8d9-4541-bcbd-af44efbcd177-host-slash\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975889    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5j9s\" (UniqueName: \"kubernetes.io/projected/1ed77e69-f8d9-4541-bcbd-af44efbcd177-kube-api-access-p5j9s\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:02.975991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975916    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-run-netns\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975941    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-var-lib-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.975989    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovnkube-script-lib\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976015    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-netns\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976030    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-cni-bin\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976044    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-hostroot\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976081    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysctl-conf\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976109    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-var-lib-kubelet\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976133    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-cnibin\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976156    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1fb37b0-fa56-4167-873a-c72bb0947ebd-kubelet-config\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976185    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysctl-conf\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976216    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysconfig\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976240    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-var-lib-kubelet\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976281    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-sysconfig\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-node-log\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.976588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976341    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-env-overrides\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976372    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-conf-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976394    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-os-release\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976413    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-etc-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976432    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95e94611-4840-4771-ac29-e1598ac1a3f4-cni-binary-copy\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976450    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1fb37b0-fa56-4167-873a-c72bb0947ebd-dbus\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976471    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-lib-modules\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtdkf\" (UniqueName: \"kubernetes.io/projected/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-kube-api-access-gtdkf\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976560    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976571    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-lib-modules\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976576    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976608    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-run\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976634    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-etc-selinux\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976655    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-os-release\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976701    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-tmp-dir\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:02.977387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-run\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976728    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976750    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovn-node-metrics-cert\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976772    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-socket-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976793    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-tuned\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976813    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-serviceca\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976842    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976897    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-socket-dir-parent\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.976953    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977003    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-etc-kubernetes\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977029    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-slash\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977057    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-device-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977082    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-sys-fs\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977110    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-system-cni-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qvwh\" (UniqueName: \"kubernetes.io/projected/95e94611-4840-4771-ac29-e1598ac1a3f4-kube-api-access-2qvwh\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.978138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977172    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-sys\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977198    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2x96\" (UniqueName: \"kubernetes.io/projected/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-kube-api-access-q2x96\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977230    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-systemd-units\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-log-socket\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977284    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9mmg\" (UniqueName: \"kubernetes.io/projected/71fb71f3-319c-45c4-bb26-88a3c19a687f-kube-api-access-d9mmg\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977305    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-cni-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovnkube-config\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977348    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9lqt\" (UniqueName: \"kubernetes.io/projected/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-kube-api-access-c9lqt\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977370    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9pgj\" (UniqueName: \"kubernetes.io/projected/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-kube-api-access-d9pgj\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.977585    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/5ffc13fa-3522-4ec9-8d85-6681936e8104-sys\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.978917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.978708    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-serviceca\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:02.979729 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.979703    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5ffc13fa-3522-4ec9-8d85-6681936e8104-tmp\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.979810 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.979764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/5ffc13fa-3522-4ec9-8d85-6681936e8104-etc-tuned\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.986402 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.986378    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vgbq\" (UniqueName: \"kubernetes.io/projected/5ffc13fa-3522-4ec9-8d85-6681936e8104-kube-api-access-2vgbq\") pod \"tuned-fcp6w\" (UID: \"5ffc13fa-3522-4ec9-8d85-6681936e8104\") " pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:02.986843 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:02.986824    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2x96\" (UniqueName: \"kubernetes.io/projected/52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8-kube-api-access-q2x96\") pod \"node-ca-szrr5\" (UID: \"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8\") " pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:03.060745 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.060704    2562 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:12:03.078368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078343    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ed77e69-f8d9-4541-bcbd-af44efbcd177-iptables-alerter-script\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:03.078368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078369    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-kubelet\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078519 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078385    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-systemd\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078519 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078399    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:03.078519 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.078477    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:03.078624 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.078522    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs podName:725501d7-58c0-4b02-9e3f-8e931b0d0dfc nodeName:}" failed. No retries permitted until 2026-07-25 01:12:03.578508421 +0000 UTC m=+2.049464737 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs") pod "network-metrics-daemon-w6nf6" (UID: "725501d7-58c0-4b02-9e3f-8e931b0d0dfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:03.078624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078515    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-registration-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.078624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078546    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-cni-multus\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.078624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078568    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-system-cni-dir\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.078624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078591    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ed77e69-f8d9-4541-bcbd-af44efbcd177-host-slash\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:03.078624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078600    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-systemd\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078615    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5j9s\" (UniqueName: \"kubernetes.io/projected/1ed77e69-f8d9-4541-bcbd-af44efbcd177-kube-api-access-p5j9s\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078628    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-registration-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078639    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-run-netns\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078637    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-cni-multus\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078663    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-var-lib-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078660    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-system-cni-dir\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078685    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/1ed77e69-f8d9-4541-bcbd-af44efbcd177-host-slash\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-var-lib-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078772    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-kubelet\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078774    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-run-netns\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078794    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovnkube-script-lib\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078811    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-netns\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078825    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-cni-bin\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.078850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-hostroot\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078868    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-cnibin\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078871    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-netns\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078888    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/1ed77e69-f8d9-4541-bcbd-af44efbcd177-iptables-alerter-script\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078876    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-cni-bin\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-hostroot\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.078967    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-cnibin\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079001    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1fb37b0-fa56-4167-873a-c72bb0947ebd-kubelet-config\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079018    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079041    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-node-log\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079066    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079068    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e1fb37b0-fa56-4167-873a-c72bb0947ebd-kubelet-config\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079092    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-env-overrides\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079111    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-node-log\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-run-ovn-kubernetes\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079115    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-conf-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079152    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-conf-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-os-release\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.079337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079169    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079179    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-etc-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079207    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95e94611-4840-4771-ac29-e1598ac1a3f4-cni-binary-copy\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079232    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1fb37b0-fa56-4167-873a-c72bb0947ebd-dbus\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovnkube-script-lib\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079256    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtdkf\" (UniqueName: \"kubernetes.io/projected/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-kube-api-access-gtdkf\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079232    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-os-release\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-etc-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079349    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079352    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e1fb37b0-fa56-4167-873a-c72bb0947ebd-dbus\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079404    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.079422    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-etc-selinux\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-os-release\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.079478    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret podName:e1fb37b0-fa56-4167-873a-c72bb0947ebd nodeName:}" failed. No retries permitted until 2026-07-25 01:12:03.579464447 +0000 UTC m=+2.050420774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret") pod "global-pull-secret-syncer-c46qr" (UID: "e1fb37b0-fa56-4167-873a-c72bb0947ebd") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079505    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079510    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-env-overrides\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.080193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079529    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-tmp-dir\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079578    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovn-node-metrics-cert\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079602    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-socket-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079624    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-etc-selinux\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079628    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-socket-dir-parent\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079687    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079714    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/95e94611-4840-4771-ac29-e1598ac1a3f4-cni-binary-copy\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079735    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-etc-kubernetes\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079531    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-os-release\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079768    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-slash\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079781    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-etc-kubernetes\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079798    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-device-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079823    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-sys-fs\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079828    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-socket-dir-parent\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079840    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-tmp-dir\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-system-cni-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079871    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qvwh\" (UniqueName: \"kubernetes.io/projected/95e94611-4840-4771-ac29-e1598ac1a3f4-kube-api-access-2qvwh\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079884    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-slash\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079898    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-systemd-units\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-openvswitch\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079923    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-log-socket\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9mmg\" (UniqueName: \"kubernetes.io/projected/71fb71f3-319c-45c4-bb26-88a3c19a687f-kube-api-access-d9mmg\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079956    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080007    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-device-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-cni-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080045    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-systemd-units\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovnkube-config\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9lqt\" (UniqueName: \"kubernetes.io/projected/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-kube-api-access-c9lqt\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9pgj\" (UniqueName: \"kubernetes.io/projected/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-kube-api-access-d9pgj\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080126    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.081852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080171    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-cni-netd\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080196    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-daemon-config\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-log-socket\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080220    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-hosts-file\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cni-binary-copy\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080244    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhvsh\" (UniqueName: \"kubernetes.io/projected/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-kube-api-access-qhvsh\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080280    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-cni-bin\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080311    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080320    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-cni-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-socket-dir\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.079947    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-sys-fs\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080096    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-system-cni-dir\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080336    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-k8s-cni-cncf-io\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080377    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-k8s-cni-cncf-io\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080386    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-cni-netd\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080410    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-kubelet\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080436    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cnibin\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-ovn\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.082724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080483    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-hosts-file\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080521    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-var-lib-kubelet\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080561    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-cnibin\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080437    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080587    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-host-cni-bin\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080485    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8237157-cb64-498e-8759-7715d46af050-agent-certs\") pod \"konnectivity-agent-8qxdx\" (UID: \"b8237157-cb64-498e-8759-7715d46af050\") " pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080607    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/71fb71f3-319c-45c4-bb26-88a3c19a687f-run-ovn\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-multus-certs\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080657    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8237157-cb64-498e-8759-7715d46af050-konnectivity-ca\") pod \"konnectivity-agent-8qxdx\" (UID: \"b8237157-cb64-498e-8759-7715d46af050\") " pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080631    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovnkube-config\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080737    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/95e94611-4840-4771-ac29-e1598ac1a3f4-host-run-multus-certs\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.080928    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/95e94611-4840-4771-ac29-e1598ac1a3f4-multus-daemon-config\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.081130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b8237157-cb64-498e-8759-7715d46af050-konnectivity-ca\") pod \"konnectivity-agent-8qxdx\" (UID: \"b8237157-cb64-498e-8759-7715d46af050\") " pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.082246    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/71fb71f3-319c-45c4-bb26-88a3c19a687f-ovn-node-metrics-cert\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.083250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.083027    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b8237157-cb64-498e-8759-7715d46af050-agent-certs\") pod \"konnectivity-agent-8qxdx\" (UID: \"b8237157-cb64-498e-8759-7715d46af050\") " pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:03.088852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.088834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5j9s\" (UniqueName: \"kubernetes.io/projected/1ed77e69-f8d9-4541-bcbd-af44efbcd177-kube-api-access-p5j9s\") pod \"iptables-alerter-fjcfh\" (UID: \"1ed77e69-f8d9-4541-bcbd-af44efbcd177\") " pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:03.089246 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.089231    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:12:03.089306 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.089250    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:12:03.089306 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.089263    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bx7tm for pod openshift-network-diagnostics/network-check-target-5fqmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:03.089392 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.089313    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm podName:500981ce-a9b7-4efb-8d4b-8f0ca9ceec63 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:03.589301382 +0000 UTC m=+2.060257711 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-bx7tm" (UniqueName: "kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm") pod "network-check-target-5fqmf" (UID: "500981ce-a9b7-4efb-8d4b-8f0ca9ceec63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:03.090781 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.090765    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtdkf\" (UniqueName: \"kubernetes.io/projected/d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e-kube-api-access-gtdkf\") pod \"aws-ebs-csi-driver-node-kjvzw\" (UID: \"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.095126 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.095097    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b5e70bad49fe9daa9a739742c8d5fb2.slice/crio-4a613c65e025200c4bec6f55158277a516712bc4d29fb3fc9da1c0416fd4e965 WatchSource:0}: Error finding container 4a613c65e025200c4bec6f55158277a516712bc4d29fb3fc9da1c0416fd4e965: Status 404 returned error can't find the container with id 4a613c65e025200c4bec6f55158277a516712bc4d29fb3fc9da1c0416fd4e965
Jul 25 01:12:03.095456 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.095432    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ed3ed7be1c2695b0c97f92390b9feef.slice/crio-696d6f3c9e6ae3568597b91a744bfdc2d495d3e7c4d5c6e8b4d6d8e546f8dff2 WatchSource:0}: Error finding container 696d6f3c9e6ae3568597b91a744bfdc2d495d3e7c4d5c6e8b4d6d8e546f8dff2: Status 404 returned error can't find the container with id 696d6f3c9e6ae3568597b91a744bfdc2d495d3e7c4d5c6e8b4d6d8e546f8dff2
Jul 25 01:12:03.096935 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.096286    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9mmg\" (UniqueName: \"kubernetes.io/projected/71fb71f3-319c-45c4-bb26-88a3c19a687f-kube-api-access-d9mmg\") pod \"ovnkube-node-s7gf8\" (UID: \"71fb71f3-319c-45c4-bb26-88a3c19a687f\") " pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.098510 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.098489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhvsh\" (UniqueName: \"kubernetes.io/projected/2f15f82d-6ba2-4275-a812-d1cd89ea9f69-kube-api-access-qhvsh\") pod \"node-resolver-hljng\" (UID: \"2f15f82d-6ba2-4275-a812-d1cd89ea9f69\") " pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:03.098867 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.098849    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9lqt\" (UniqueName: \"kubernetes.io/projected/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-kube-api-access-c9lqt\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:03.100263 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.100245    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qvwh\" (UniqueName: \"kubernetes.io/projected/95e94611-4840-4771-ac29-e1598ac1a3f4-kube-api-access-2qvwh\") pod \"multus-wwkc9\" (UID: \"95e94611-4840-4771-ac29-e1598ac1a3f4\") " pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.100383 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.100368    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9pgj\" (UniqueName: \"kubernetes.io/projected/dd88ee59-a0f7-4608-8c0a-77cc79d6e25b-kube-api-access-d9pgj\") pod \"multus-additional-cni-plugins-vl7nz\" (UID: \"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b\") " pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.101417 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.101402    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:12:03.196476 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.196393    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fcp6w"
Jul 25 01:12:03.201749 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.201724    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-szrr5"
Jul 25 01:12:03.205994 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.205952    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ffc13fa_3522_4ec9_8d85_6681936e8104.slice/crio-2a2ed95f6c8bb541336cbb922139e81be7737bbfc85486bee1a0bfc7e9521969 WatchSource:0}: Error finding container 2a2ed95f6c8bb541336cbb922139e81be7737bbfc85486bee1a0bfc7e9521969: Status 404 returned error can't find the container with id 2a2ed95f6c8bb541336cbb922139e81be7737bbfc85486bee1a0bfc7e9521969
Jul 25 01:12:03.210886 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.210860    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52431c6d_7b58_4ecb_b1d1_d39ad37f1fd8.slice/crio-30a26852c52a821e4282b76ebe61c501b134759fac31724027adb5c8bfc6aae9 WatchSource:0}: Error finding container 30a26852c52a821e4282b76ebe61c501b134759fac31724027adb5c8bfc6aae9: Status 404 returned error can't find the container with id 30a26852c52a821e4282b76ebe61c501b134759fac31724027adb5c8bfc6aae9
Jul 25 01:12:03.238605 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.238580    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vl7nz"
Jul 25 01:12:03.241956 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.241925    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fjcfh"
Jul 25 01:12:03.247787 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.247757    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd88ee59_a0f7_4608_8c0a_77cc79d6e25b.slice/crio-fc98357002d39e202db0f0e8897115c171ed559517423bc68414f3d7b6bd0b30 WatchSource:0}: Error finding container fc98357002d39e202db0f0e8897115c171ed559517423bc68414f3d7b6bd0b30: Status 404 returned error can't find the container with id fc98357002d39e202db0f0e8897115c171ed559517423bc68414f3d7b6bd0b30
Jul 25 01:12:03.250598 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.250578    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ed77e69_f8d9_4541_bcbd_af44efbcd177.slice/crio-948bf1a4121110710e44d3784ed79786a68d29496571195b941b192fad56ad1b WatchSource:0}: Error finding container 948bf1a4121110710e44d3784ed79786a68d29496571195b941b192fad56ad1b: Status 404 returned error can't find the container with id 948bf1a4121110710e44d3784ed79786a68d29496571195b941b192fad56ad1b
Jul 25 01:12:03.263919 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.263896    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:03.272208 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.272188    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71fb71f3_319c_45c4_bb26_88a3c19a687f.slice/crio-55bae4e68f40e2527b806e0e9c2d89dcaebbd028127684f2576105ae34f00a32 WatchSource:0}: Error finding container 55bae4e68f40e2527b806e0e9c2d89dcaebbd028127684f2576105ae34f00a32: Status 404 returned error can't find the container with id 55bae4e68f40e2527b806e0e9c2d89dcaebbd028127684f2576105ae34f00a32
Jul 25 01:12:03.280888 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.280873    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:03.288741 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.288682    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8237157_cb64_498e_8759_7715d46af050.slice/crio-295bd737e0f9667628995e3b517c7c4f11806dd0118c2ec39dcadcaa60ad5377 WatchSource:0}: Error finding container 295bd737e0f9667628995e3b517c7c4f11806dd0118c2ec39dcadcaa60ad5377: Status 404 returned error can't find the container with id 295bd737e0f9667628995e3b517c7c4f11806dd0118c2ec39dcadcaa60ad5377
Jul 25 01:12:03.308109 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.308080    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw"
Jul 25 01:12:03.316057 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.316034    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7d1caf4_156d_4cbe_a9d1_b0b058e3cf9e.slice/crio-10a4708616cedb4eccc0a9f3bb021bf404e29068ae364e0bed61d7387e28fe86 WatchSource:0}: Error finding container 10a4708616cedb4eccc0a9f3bb021bf404e29068ae364e0bed61d7387e28fe86: Status 404 returned error can't find the container with id 10a4708616cedb4eccc0a9f3bb021bf404e29068ae364e0bed61d7387e28fe86
Jul 25 01:12:03.329340 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.329317    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hljng"
Jul 25 01:12:03.333110 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.333096    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wwkc9"
Jul 25 01:12:03.337228 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.337208    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f15f82d_6ba2_4275_a812_d1cd89ea9f69.slice/crio-c529d5eb4fc20d1041c01fe4bd55e78290a5f51b2e17a49ec316eac1e5828b39 WatchSource:0}: Error finding container c529d5eb4fc20d1041c01fe4bd55e78290a5f51b2e17a49ec316eac1e5828b39: Status 404 returned error can't find the container with id c529d5eb4fc20d1041c01fe4bd55e78290a5f51b2e17a49ec316eac1e5828b39
Jul 25 01:12:03.340445 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:03.340423    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod95e94611_4840_4771_ac29_e1598ac1a3f4.slice/crio-3188b66145da4d79602036248074158ab960315845cc683753d6b381ff0e6bcd WatchSource:0}: Error finding container 3188b66145da4d79602036248074158ab960315845cc683753d6b381ff0e6bcd: Status 404 returned error can't find the container with id 3188b66145da4d79602036248074158ab960315845cc683753d6b381ff0e6bcd
Jul 25 01:12:03.584713 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.584091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:03.584713 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.584155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:03.584713 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.584287    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:03.584713 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.584343    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret podName:e1fb37b0-fa56-4167-873a-c72bb0947ebd nodeName:}" failed. No retries permitted until 2026-07-25 01:12:04.584324121 +0000 UTC m=+3.055280438 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret") pod "global-pull-secret-syncer-c46qr" (UID: "e1fb37b0-fa56-4167-873a-c72bb0947ebd") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:03.584713 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.584551    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:03.584713 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.584638    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs podName:725501d7-58c0-4b02-9e3f-8e931b0d0dfc nodeName:}" failed. No retries permitted until 2026-07-25 01:12:04.584618045 +0000 UTC m=+3.055574369 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs") pod "network-metrics-daemon-w6nf6" (UID: "725501d7-58c0-4b02-9e3f-8e931b0d0dfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:03.684773 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.684740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:03.684951 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.684911    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:12:03.684951 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.684932    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:12:03.684951 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.684945    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bx7tm for pod openshift-network-diagnostics/network-check-target-5fqmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:03.685189 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:03.685015    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm podName:500981ce-a9b7-4efb-8d4b-8f0ca9ceec63 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:04.684995418 +0000 UTC m=+3.155951734 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-bx7tm" (UniqueName: "kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm") pod "network-check-target-5fqmf" (UID: "500981ce-a9b7-4efb-8d4b-8f0ca9ceec63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:03.825233 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.825173    2562 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:12:03.875306 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.875279    2562 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:12:03.917392 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.917304    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-24 01:07:02 +0000 UTC" deadline="2028-04-14 00:07:38.368147176 +0000 UTC"
Jul 25 01:12:03.917392 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:03.917367    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15094h55m34.450784273s"
Jul 25 01:12:04.077947 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.077920    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:04.078149 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.078057    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:04.078481 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.078440    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:04.078572 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.078526    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:04.107046 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.106861    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hljng" event={"ID":"2f15f82d-6ba2-4275-a812-d1cd89ea9f69","Type":"ContainerStarted","Data":"c529d5eb4fc20d1041c01fe4bd55e78290a5f51b2e17a49ec316eac1e5828b39"}
Jul 25 01:12:04.124837 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.124796    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8qxdx" event={"ID":"b8237157-cb64-498e-8759-7715d46af050","Type":"ContainerStarted","Data":"295bd737e0f9667628995e3b517c7c4f11806dd0118c2ec39dcadcaa60ad5377"}
Jul 25 01:12:04.130817 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.130653    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"55bae4e68f40e2527b806e0e9c2d89dcaebbd028127684f2576105ae34f00a32"}
Jul 25 01:12:04.151380 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.151345    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fjcfh" event={"ID":"1ed77e69-f8d9-4541-bcbd-af44efbcd177","Type":"ContainerStarted","Data":"948bf1a4121110710e44d3784ed79786a68d29496571195b941b192fad56ad1b"}
Jul 25 01:12:04.174924 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.174885    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal" event={"ID":"2b5e70bad49fe9daa9a739742c8d5fb2","Type":"ContainerStarted","Data":"4a613c65e025200c4bec6f55158277a516712bc4d29fb3fc9da1c0416fd4e965"}
Jul 25 01:12:04.205912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.205843    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wwkc9" event={"ID":"95e94611-4840-4771-ac29-e1598ac1a3f4","Type":"ContainerStarted","Data":"3188b66145da4d79602036248074158ab960315845cc683753d6b381ff0e6bcd"}
Jul 25 01:12:04.217303 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.217269    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw" event={"ID":"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e","Type":"ContainerStarted","Data":"10a4708616cedb4eccc0a9f3bb021bf404e29068ae364e0bed61d7387e28fe86"}
Jul 25 01:12:04.233230 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.233195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerStarted","Data":"fc98357002d39e202db0f0e8897115c171ed559517423bc68414f3d7b6bd0b30"}
Jul 25 01:12:04.238185 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.238154    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-szrr5" event={"ID":"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8","Type":"ContainerStarted","Data":"30a26852c52a821e4282b76ebe61c501b134759fac31724027adb5c8bfc6aae9"}
Jul 25 01:12:04.245216 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.244428    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fcp6w" event={"ID":"5ffc13fa-3522-4ec9-8d85-6681936e8104","Type":"ContainerStarted","Data":"2a2ed95f6c8bb541336cbb922139e81be7737bbfc85486bee1a0bfc7e9521969"}
Jul 25 01:12:04.246009 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.245959    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal" event={"ID":"1ed3ed7be1c2695b0c97f92390b9feef","Type":"ContainerStarted","Data":"696d6f3c9e6ae3568597b91a744bfdc2d495d3e7c4d5c6e8b4d6d8e546f8dff2"}
Jul 25 01:12:04.590715 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.590638    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:04.590715 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.590699    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:04.590936 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.590834    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:04.590936 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.590897    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret podName:e1fb37b0-fa56-4167-873a-c72bb0947ebd nodeName:}" failed. No retries permitted until 2026-07-25 01:12:06.590878439 +0000 UTC m=+5.061834757 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret") pod "global-pull-secret-syncer-c46qr" (UID: "e1fb37b0-fa56-4167-873a-c72bb0947ebd") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:04.591345 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.591293    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:04.591432 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.591348    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs podName:725501d7-58c0-4b02-9e3f-8e931b0d0dfc nodeName:}" failed. No retries permitted until 2026-07-25 01:12:06.591333242 +0000 UTC m=+5.062289563 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs") pod "network-metrics-daemon-w6nf6" (UID: "725501d7-58c0-4b02-9e3f-8e931b0d0dfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:04.691776 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.691743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:04.691939 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.691918    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:12:04.691939 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.691937    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:12:04.692126 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.691949    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bx7tm for pod openshift-network-diagnostics/network-check-target-5fqmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:04.692126 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:04.692015    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm podName:500981ce-a9b7-4efb-8d4b-8f0ca9ceec63 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:06.691998046 +0000 UTC m=+5.162954363 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-bx7tm" (UniqueName: "kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm") pod "network-check-target-5fqmf" (UID: "500981ce-a9b7-4efb-8d4b-8f0ca9ceec63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:04.918469 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.918428    2562 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-24 01:07:02 +0000 UTC" deadline="2027-12-22 15:24:04.36188542 +0000 UTC"
Jul 25 01:12:04.918469 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:04.918467    2562 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12374h11m59.443423283s"
Jul 25 01:12:05.070579 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:05.070200    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:05.070579 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:05.070309    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:05.193875 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:05.193797    2562 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Jul 25 01:12:06.084906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:06.084827    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:06.085383 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:06.085072    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:06.085383 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.085174    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:06.086054 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.085970    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:06.609726 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:06.609691    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:06.609894 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:06.609757    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:06.609894 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.609867    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:06.609997 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.609910    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:06.609997 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.609938    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs podName:725501d7-58c0-4b02-9e3f-8e931b0d0dfc nodeName:}" failed. No retries permitted until 2026-07-25 01:12:10.609918101 +0000 UTC m=+9.080874419 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs") pod "network-metrics-daemon-w6nf6" (UID: "725501d7-58c0-4b02-9e3f-8e931b0d0dfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:06.609997 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.609961    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret podName:e1fb37b0-fa56-4167-873a-c72bb0947ebd nodeName:}" failed. No retries permitted until 2026-07-25 01:12:10.609945652 +0000 UTC m=+9.080901971 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret") pod "global-pull-secret-syncer-c46qr" (UID: "e1fb37b0-fa56-4167-873a-c72bb0947ebd") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:06.711011 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:06.710966    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:06.711182 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.711123    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:12:06.711182 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.711151    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:12:06.711182 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.711163    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bx7tm for pod openshift-network-diagnostics/network-check-target-5fqmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:06.711357 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:06.711223    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm podName:500981ce-a9b7-4efb-8d4b-8f0ca9ceec63 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:10.711204379 +0000 UTC m=+9.182160698 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-bx7tm" (UniqueName: "kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm") pod "network-check-target-5fqmf" (UID: "500981ce-a9b7-4efb-8d4b-8f0ca9ceec63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:07.069766 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:07.069727    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:07.069946 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:07.069878    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:08.071858 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:08.069788    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:08.071858 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:08.069914    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:08.071858 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:08.070275    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:08.071858 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:08.070362    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:09.070271 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:09.069878    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:09.070271 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:09.070046    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:10.073477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:10.073441    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:10.073955 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.073571    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:10.074149 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:10.074130    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:10.074245 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.074225    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:10.644743 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:10.644706    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:10.644903 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:10.644782    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:10.644974 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.644892    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:10.644974 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.644941    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:10.644974 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.644973    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs podName:725501d7-58c0-4b02-9e3f-8e931b0d0dfc nodeName:}" failed. No retries permitted until 2026-07-25 01:12:18.644951395 +0000 UTC m=+17.115907714 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs") pod "network-metrics-daemon-w6nf6" (UID: "725501d7-58c0-4b02-9e3f-8e931b0d0dfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:10.645168 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.645042    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret podName:e1fb37b0-fa56-4167-873a-c72bb0947ebd nodeName:}" failed. No retries permitted until 2026-07-25 01:12:18.645025087 +0000 UTC m=+17.115981412 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret") pod "global-pull-secret-syncer-c46qr" (UID: "e1fb37b0-fa56-4167-873a-c72bb0947ebd") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:10.746488 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:10.745883    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:10.746488 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.746071    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:12:10.746488 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.746093    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:12:10.746488 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.746107    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bx7tm for pod openshift-network-diagnostics/network-check-target-5fqmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:10.746488 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:10.746163    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm podName:500981ce-a9b7-4efb-8d4b-8f0ca9ceec63 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:18.74614423 +0000 UTC m=+17.217100549 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-bx7tm" (UniqueName: "kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm") pod "network-check-target-5fqmf" (UID: "500981ce-a9b7-4efb-8d4b-8f0ca9ceec63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:11.070171 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:11.070094    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:11.070318 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:11.070254    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:12.071718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:12.071261    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:12.071718 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:12.071377    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:12.071718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:12.071464    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:12.071718 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:12.071576    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:13.070451 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:13.070417    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:13.070628 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:13.070541    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:14.070229 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:14.070180    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:14.070229 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:14.070209    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:14.070729 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:14.070325    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:14.070729 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:14.070453    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:15.070558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:15.070529    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:15.071024 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:15.070658    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:16.070039 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:16.070000    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:16.070220 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:16.070045    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:16.070220 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:16.070129    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:16.070333 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:16.070260    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:17.069843 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:17.069811    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:17.070259 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:17.069937    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:18.070346 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:18.070303    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:18.070346 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:18.070345    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:18.070821 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.070443    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:18.070821 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.070573    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:18.702888 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:18.702803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:18.702888 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:18.702865    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:18.703211 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.702991    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:18.703211 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.703046    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:18.703211 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.703078    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs podName:725501d7-58c0-4b02-9e3f-8e931b0d0dfc nodeName:}" failed. No retries permitted until 2026-07-25 01:12:34.703054638 +0000 UTC m=+33.174010967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs") pod "network-metrics-daemon-w6nf6" (UID: "725501d7-58c0-4b02-9e3f-8e931b0d0dfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:18.703211 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.703101    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret podName:e1fb37b0-fa56-4167-873a-c72bb0947ebd nodeName:}" failed. No retries permitted until 2026-07-25 01:12:34.703091173 +0000 UTC m=+33.174047490 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret") pod "global-pull-secret-syncer-c46qr" (UID: "e1fb37b0-fa56-4167-873a-c72bb0947ebd") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:18.804067 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:18.804028    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:18.804244 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.804221    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:12:18.804304 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.804254    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:12:18.804304 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.804267    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bx7tm for pod openshift-network-diagnostics/network-check-target-5fqmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:18.804372 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:18.804319    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm podName:500981ce-a9b7-4efb-8d4b-8f0ca9ceec63 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:34.804306304 +0000 UTC m=+33.275262621 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-bx7tm" (UniqueName: "kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm") pod "network-check-target-5fqmf" (UID: "500981ce-a9b7-4efb-8d4b-8f0ca9ceec63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:19.069595 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:19.069516    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:19.069815 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:19.069666    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:20.070096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:20.070059    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:20.070551 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:20.070059    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:20.070551 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:20.070182    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:20.070551 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:20.070259    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:21.069727 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:21.069693    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:21.069906 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:21.069818    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:22.070706 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:22.070660    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:22.071182 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:22.070772    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:22.071182 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:22.070838    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:22.071182 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:22.070920    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:23.069879 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.069693    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:23.070035 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:23.069934    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:23.291724 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.291551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wwkc9" event={"ID":"95e94611-4840-4771-ac29-e1598ac1a3f4","Type":"ContainerStarted","Data":"fae24ab7abbb58f6df2cbdd4f428d53dd9f18a25a27d639f079bc0c18689006e"}
Jul 25 01:12:23.293223 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.293191    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fcp6w" event={"ID":"5ffc13fa-3522-4ec9-8d85-6681936e8104","Type":"ContainerStarted","Data":"d0a57b52dc88cd5665cdda379655763a4581fad993f591fa5a422a6e3dabfac0"}
Jul 25 01:12:23.294522 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.294497    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal" event={"ID":"1ed3ed7be1c2695b0c97f92390b9feef","Type":"ContainerStarted","Data":"4156bd619631499df738989433d08a68475924bcf360450d6bd03b9cc2bc2121"}
Jul 25 01:12:23.296099 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.296074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"1a2e5c4bbf4c51c1681d97cec7e059861767df7032777e6c8b8edce4d3b82752"}
Jul 25 01:12:23.296186 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.296140    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"18dfebf09395eb3e1cbb73fb297ab5d0641e79a7568a70a5301f540498bd1b46"}
Jul 25 01:12:23.330410 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.330363    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-wwkc9" podStartSLOduration=1.8773617360000001 podStartE2EDuration="21.330347945s" totalImagesPullingTime="19.452986209s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:23.329716036 +0000 UTC m=+21.800672376" watchObservedRunningTime="2026-07-25 01:12:23.330347945 +0000 UTC m=+21.801304283"
Jul 25 01:12:23.342641 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.342598    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-129-103.ec2.internal" podStartSLOduration=21.342586179 podStartE2EDuration="21.342586179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:23.342375541 +0000 UTC m=+21.813331879" watchObservedRunningTime="2026-07-25 01:12:23.342586179 +0000 UTC m=+21.813542517"
Jul 25 01:12:23.358569 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:23.358532    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fcp6w" podStartSLOduration=1.786405685 podStartE2EDuration="21.358519227s" totalImagesPullingTime="19.572113542s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:23.358376219 +0000 UTC m=+21.829332554" watchObservedRunningTime="2026-07-25 01:12:23.358519227 +0000 UTC m=+21.829475564"
Jul 25 01:12:24.069946 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.069924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:24.070075 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:24.070057    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:24.070123 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.070089    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:24.070157 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:24.070137    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:24.299171 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.299136    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw" event={"ID":"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e","Type":"ContainerStarted","Data":"ebef03e42d37fdf26fa723e76eb9596be0f77c1d3cd970d6a0c67e6ba79357c8"}
Jul 25 01:12:24.300488 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.300466    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd88ee59-a0f7-4608-8c0a-77cc79d6e25b" containerID="4a01b493f49b4305438a265b7751b45e87c19d8be79394c7def9086a4c4e8423" exitCode=0
Jul 25 01:12:24.300580 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.300530    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerDied","Data":"4a01b493f49b4305438a265b7751b45e87c19d8be79394c7def9086a4c4e8423"}
Jul 25 01:12:24.301954 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.301930    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-szrr5" event={"ID":"52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8","Type":"ContainerStarted","Data":"4e03a62f8dd8a3d77324a3cf20c650a228bf63b915d1acc83bc315a02ab531a0"}
Jul 25 01:12:24.303254 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.303235    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hljng" event={"ID":"2f15f82d-6ba2-4275-a812-d1cd89ea9f69","Type":"ContainerStarted","Data":"31ce0100c101bd4f82d4657d981ade7e657cd7e862b23ca25b497c843ed732d5"}
Jul 25 01:12:24.304688 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.304638    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8qxdx" event={"ID":"b8237157-cb64-498e-8759-7715d46af050","Type":"ContainerStarted","Data":"efd02291990a9c9a9da2dc50d6221cfdb7e6286ddc1309b619e4c9be5bf532bc"}
Jul 25 01:12:24.307439 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.307423    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"41c2c6578b1305af86c9fc3520be22d606fb2c85dc6b9b9b5eb0f2c56a3ce684"}
Jul 25 01:12:24.307519 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.307473    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"e1f2c0ce217bda21fe256cc57aecdf3ae366c02b223529592ac32d8cad1e93e9"}
Jul 25 01:12:24.307519 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.307511    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"717284a2a01a409c5db21e7f200d61d52180befe41e4d7d3b8cda925b60742bd"}
Jul 25 01:12:24.307640 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.307540    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"6f89c09c8cc24357e3bc4b95b2df17b8f954f82f7d24bbd2e8fd357941945dba"}
Jul 25 01:12:24.308715 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.308697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fjcfh" event={"ID":"1ed77e69-f8d9-4541-bcbd-af44efbcd177","Type":"ContainerStarted","Data":"be6cfb9dc46935796bf2613281c30920336713c14d1ee13a489fdafa3ca8a758"}
Jul 25 01:12:24.309993 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.309915    2562 generic.go:356] "Generic (PLEG): container finished" podID="2b5e70bad49fe9daa9a739742c8d5fb2" containerID="e7274629406ee7f7f316977c12f631998499c8954f6c96147c9511c86aff13ae" exitCode=0
Jul 25 01:12:24.309993 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.309989    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal" event={"ID":"2b5e70bad49fe9daa9a739742c8d5fb2","Type":"ContainerDied","Data":"e7274629406ee7f7f316977c12f631998499c8954f6c96147c9511c86aff13ae"}
Jul 25 01:12:24.336728 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.336690    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8qxdx" podStartSLOduration=2.893987161 podStartE2EDuration="22.336678591s" totalImagesPullingTime="19.44269143s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:24.336546457 +0000 UTC m=+22.807502795" watchObservedRunningTime="2026-07-25 01:12:24.336678591 +0000 UTC m=+22.807634929"
Jul 25 01:12:24.365586 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.365546    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-szrr5" podStartSLOduration=2.798153751 podStartE2EDuration="22.36553021s" totalImagesPullingTime="19.567376459s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:24.365490957 +0000 UTC m=+22.836447294" watchObservedRunningTime="2026-07-25 01:12:24.36553021 +0000 UTC m=+22.836486526"
Jul 25 01:12:24.365968 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.365945    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hljng" podStartSLOduration=2.927093857 podStartE2EDuration="22.365939846s" totalImagesPullingTime="19.438845989s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:24.350468869 +0000 UTC m=+22.821425207" watchObservedRunningTime="2026-07-25 01:12:24.365939846 +0000 UTC m=+22.836896183"
Jul 25 01:12:24.380742 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.380701    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fjcfh" podStartSLOduration=2.899883988 podStartE2EDuration="22.380693064s" totalImagesPullingTime="19.480809076s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:24.380593972 +0000 UTC m=+22.851550339" watchObservedRunningTime="2026-07-25 01:12:24.380693064 +0000 UTC m=+22.851649401"
Jul 25 01:12:24.865263 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.865094    2562 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Jul 25 01:12:24.944403 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.944287    2562 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-25T01:12:24.865245623Z","UUID":"d648d110-17f1-4d59-a1e0-7df31ba9694b","Handler":null,"Name":"","Endpoint":""}
Jul 25 01:12:24.945835 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.945814    2562 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Jul 25 01:12:24.945955 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:24.945860    2562 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Jul 25 01:12:25.069880 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:25.069851    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:25.070079 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:25.069974    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:25.314792 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:25.314748    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw" event={"ID":"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e","Type":"ContainerStarted","Data":"92cbb906affa9f6dea16fc9e2082285f612a77ee1347da3818b5a463c64ad3fa"}
Jul 25 01:12:25.317147 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:25.317072    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal" event={"ID":"2b5e70bad49fe9daa9a739742c8d5fb2","Type":"ContainerStarted","Data":"8870704b5f48f516fce06ac86bb611b315542b2c6ee13463b5afc4a70ee9d7bc"}
Jul 25 01:12:25.333808 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:25.333768    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-129-103.ec2.internal" podStartSLOduration=23.333751504 podStartE2EDuration="23.333751504s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:25.332887754 +0000 UTC m=+23.803844092" watchObservedRunningTime="2026-07-25 01:12:25.333751504 +0000 UTC m=+23.804707841"
Jul 25 01:12:26.071013 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:26.070970    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:26.071013 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:26.071002    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:26.071175 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:26.071117    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:26.071501 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:26.071479    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:26.323540 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:26.323454    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"30f647c4771ca24b093b07749b99e013d8d9c07245aaaa4df32d60fc6116cd87"}
Jul 25 01:12:26.325741 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:26.325708    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw" event={"ID":"d7d1caf4-156d-4cbe-a9d1-b0b058e3cf9e","Type":"ContainerStarted","Data":"a58d5d495e3bd0f63be582dbcb61ef16e21039ebb1e7c6d6678f1bbc0d227441"}
Jul 25 01:12:26.344203 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:26.344162    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kjvzw" podStartSLOduration=2.727214886 podStartE2EDuration="24.344149184s" totalImagesPullingTime="21.616934298s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:26.344052504 +0000 UTC m=+24.815008845" watchObservedRunningTime="2026-07-25 01:12:26.344149184 +0000 UTC m=+24.815105521"
Jul 25 01:12:27.069971 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:27.069937    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:27.070201 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:27.070090    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:28.070398 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:28.070370    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:28.070906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:28.070379    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:28.070906 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:28.070486    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:28.070906 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:28.070596    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:29.070564 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.070374    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:29.071074 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:29.070658    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:29.114228 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.114204    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:29.114777 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.114759    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:29.335703 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.335635    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" event={"ID":"71fb71f3-319c-45c4-bb26-88a3c19a687f","Type":"ContainerStarted","Data":"ecfaee2ee7d67e6f9f6c860cf6edf3d065d1ceba9dd2f9e46df3bebd5c801dd9"}
Jul 25 01:12:29.335906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.335868    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:29.337495 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.337472    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd88ee59-a0f7-4608-8c0a-77cc79d6e25b" containerID="29e3755f8d1a41d103f24f369331aaa165952b7e204c5cda23059c75a0bf154a" exitCode=0
Jul 25 01:12:29.337621 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.337505    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerDied","Data":"29e3755f8d1a41d103f24f369331aaa165952b7e204c5cda23059c75a0bf154a"}
Jul 25 01:12:29.337723 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.337709    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:29.338133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.338108    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8qxdx"
Jul 25 01:12:29.350817 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.350795    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:29.367627 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:29.367586    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8" podStartSLOduration=7.550896525 podStartE2EDuration="27.367573467s" totalImagesPullingTime="19.816676942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:29.36388487 +0000 UTC m=+27.834841207" watchObservedRunningTime="2026-07-25 01:12:29.367573467 +0000 UTC m=+27.838529804"
Jul 25 01:12:30.070331 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.070300    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:30.070497 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:30.070434    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:30.070497 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.070482    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:30.070591 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:30.070572    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:30.342513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.342479    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd88ee59-a0f7-4608-8c0a-77cc79d6e25b" containerID="666e81e315803b0b7914e397ae1bccc7cd658d6232b5794c3320afba61f03b8e" exitCode=0
Jul 25 01:12:30.342645 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.342565    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerDied","Data":"666e81e315803b0b7914e397ae1bccc7cd658d6232b5794c3320afba61f03b8e"}
Jul 25 01:12:30.342707 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.342692    2562 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Jul 25 01:12:30.343113 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.343082    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:30.359461 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.359443    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:30.579420 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.579309    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5fqmf"]
Jul 25 01:12:30.579573 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.579432    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:30.579573 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:30.579551    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:30.580096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.580074    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w6nf6"]
Jul 25 01:12:30.580225 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.580179    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:30.580314 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:30.580294    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:30.580736 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.580715    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c46qr"]
Jul 25 01:12:30.580848 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:30.580788    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:30.580908 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:30.580871    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:31.219486 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:31.219455    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:12:32.071035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:32.071006    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:32.071191 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:32.071064    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:32.071191 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:32.071077    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:32.071191 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:32.071151    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:32.071288 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:32.071197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:32.071288 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:32.071252    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:32.348884 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:32.348804    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd88ee59-a0f7-4608-8c0a-77cc79d6e25b" containerID="84b3720322996d062f909ff8359ecc6b33a12d90b4449ff52ad49d36a91a12ae" exitCode=0
Jul 25 01:12:32.349240 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:32.348878    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerDied","Data":"84b3720322996d062f909ff8359ecc6b33a12d90b4449ff52ad49d36a91a12ae"}
Jul 25 01:12:33.745282 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:33.745238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:33.828868 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:33.828835    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:34.070233 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:34.070042    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:34.070233 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:34.070057    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:34.070233 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:34.070057    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:34.070233 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.070148    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:34.070490 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.070236    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:34.070490 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.070337    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:34.283549 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:34.283506    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:34.337923 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:34.337840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:34.728469 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:34.728431    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:34.728621 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:34.728530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:34.728621 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.728595    2562 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:34.728709 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.728641    2562 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:34.728709 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.728662    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret podName:e1fb37b0-fa56-4167-873a-c72bb0947ebd nodeName:}" failed. No retries permitted until 2026-07-25 01:13:06.728644789 +0000 UTC m=+65.199601106 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret") pod "global-pull-secret-syncer-c46qr" (UID: "e1fb37b0-fa56-4167-873a-c72bb0947ebd") : object "kube-system"/"original-pull-secret" not registered
Jul 25 01:12:34.728709 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.728695    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs podName:725501d7-58c0-4b02-9e3f-8e931b0d0dfc nodeName:}" failed. No retries permitted until 2026-07-25 01:13:06.728677817 +0000 UTC m=+65.199634135 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs") pod "network-metrics-daemon-w6nf6" (UID: "725501d7-58c0-4b02-9e3f-8e931b0d0dfc") : object "openshift-multus"/"metrics-daemon-secret" not registered
Jul 25 01:12:34.829883 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:34.829849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:34.830329 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.830019    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Jul 25 01:12:34.830329 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.830041    2562 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Jul 25 01:12:34.830329 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.830054    2562 projected.go:196] Error preparing data for projected volume kube-api-access-bx7tm for pod openshift-network-diagnostics/network-check-target-5fqmf: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:34.830329 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:34.830115    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm podName:500981ce-a9b7-4efb-8d4b-8f0ca9ceec63 nodeName:}" failed. No retries permitted until 2026-07-25 01:13:06.83009579 +0000 UTC m=+65.301052107 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-bx7tm" (UniqueName: "kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm") pod "network-check-target-5fqmf" (UID: "500981ce-a9b7-4efb-8d4b-8f0ca9ceec63") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Jul 25 01:12:35.301325 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:35.301246    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:35.349468 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:35.349436    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:36.070480 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.070447    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:36.070931 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.070549    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:36.070931 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:36.070569    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-c46qr" podUID="e1fb37b0-fa56-4167-873a-c72bb0947ebd"
Jul 25 01:12:36.070931 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:36.070669    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-w6nf6" podUID="725501d7-58c0-4b02-9e3f-8e931b0d0dfc"
Jul 25 01:12:36.070931 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.070746    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:36.070931 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:36.070852    2562 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-5fqmf" podUID="500981ce-a9b7-4efb-8d4b-8f0ca9ceec63"
Jul 25 01:12:36.360439 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.360410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerStarted","Data":"360f9e036e922afe6a794ec68150fb082bb8b6b95ed11a72df7d2c8cf3f8faab"}
Jul 25 01:12:36.887033 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.886847    2562 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-129-103.ec2.internal" event="NodeReady"
Jul 25 01:12:36.887199 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.887115    2562 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Jul 25 01:12:36.931894 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.931865    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-b6rzf"]
Jul 25 01:12:36.934647 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.934632    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-svbfp"]
Jul 25 01:12:36.934765 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.934749    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:36.938391 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.938358    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Jul 25 01:12:36.938491 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.938374    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Jul 25 01:12:36.939131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.939108    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-fxvm2\""
Jul 25 01:12:36.939245 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.939134    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-svbfp"
Jul 25 01:12:36.941780 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.941757    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-krsnh\""
Jul 25 01:12:36.942157 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.942142    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Jul 25 01:12:36.942315 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.942155    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Jul 25 01:12:36.942624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.942606    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Jul 25 01:12:36.944033 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.944016    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b6rzf"]
Jul 25 01:12:36.956622 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:36.956604    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-svbfp"]
Jul 25 01:12:37.044608 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.044589    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb972c6d-e9ed-4b05-8817-1bed95eaae32-config-volume\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.044702 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.044613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8s6d\" (UniqueName: \"kubernetes.io/projected/bb972c6d-e9ed-4b05-8817-1bed95eaae32-kube-api-access-k8s6d\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.044702 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.044645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb972c6d-e9ed-4b05-8817-1bed95eaae32-tmp-dir\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.044702 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.044694    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.044800 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.044719    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bd143fd-07cf-4981-981c-7f515ac8336f-cert\") pod \"ingress-canary-svbfp\" (UID: \"5bd143fd-07cf-4981-981c-7f515ac8336f\") " pod="openshift-ingress-canary/ingress-canary-svbfp"
Jul 25 01:12:37.044800 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.044743    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fhhlb\" (UniqueName: \"kubernetes.io/projected/5bd143fd-07cf-4981-981c-7f515ac8336f-kube-api-access-fhhlb\") pod \"ingress-canary-svbfp\" (UID: \"5bd143fd-07cf-4981-981c-7f515ac8336f\") " pod="openshift-ingress-canary/ingress-canary-svbfp"
Jul 25 01:12:37.146065 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.146006    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb972c6d-e9ed-4b05-8817-1bed95eaae32-config-volume\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.146065 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.146037    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8s6d\" (UniqueName: \"kubernetes.io/projected/bb972c6d-e9ed-4b05-8817-1bed95eaae32-kube-api-access-k8s6d\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.146446 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.146084    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb972c6d-e9ed-4b05-8817-1bed95eaae32-tmp-dir\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.146446 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.146110    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.146446 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.146132    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bd143fd-07cf-4981-981c-7f515ac8336f-cert\") pod \"ingress-canary-svbfp\" (UID: \"5bd143fd-07cf-4981-981c-7f515ac8336f\") " pod="openshift-ingress-canary/ingress-canary-svbfp"
Jul 25 01:12:37.146446 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.146155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fhhlb\" (UniqueName: \"kubernetes.io/projected/5bd143fd-07cf-4981-981c-7f515ac8336f-kube-api-access-fhhlb\") pod \"ingress-canary-svbfp\" (UID: \"5bd143fd-07cf-4981-981c-7f515ac8336f\") " pod="openshift-ingress-canary/ingress-canary-svbfp"
Jul 25 01:12:37.146446 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:37.146249    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:12:37.146446 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:37.146320    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls podName:bb972c6d-e9ed-4b05-8817-1bed95eaae32 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:37.646304612 +0000 UTC m=+36.117260933 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls") pod "dns-default-b6rzf" (UID: "bb972c6d-e9ed-4b05-8817-1bed95eaae32") : secret "dns-default-metrics-tls" not found
Jul 25 01:12:37.146789 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.146447    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bb972c6d-e9ed-4b05-8817-1bed95eaae32-tmp-dir\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.146789 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.146588    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bb972c6d-e9ed-4b05-8817-1bed95eaae32-config-volume\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.150550 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.150528    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bd143fd-07cf-4981-981c-7f515ac8336f-cert\") pod \"ingress-canary-svbfp\" (UID: \"5bd143fd-07cf-4981-981c-7f515ac8336f\") " pod="openshift-ingress-canary/ingress-canary-svbfp"
Jul 25 01:12:37.156414 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.156391    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fhhlb\" (UniqueName: \"kubernetes.io/projected/5bd143fd-07cf-4981-981c-7f515ac8336f-kube-api-access-fhhlb\") pod \"ingress-canary-svbfp\" (UID: \"5bd143fd-07cf-4981-981c-7f515ac8336f\") " pod="openshift-ingress-canary/ingress-canary-svbfp"
Jul 25 01:12:37.156811 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.156796    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8s6d\" (UniqueName: \"kubernetes.io/projected/bb972c6d-e9ed-4b05-8817-1bed95eaae32-kube-api-access-k8s6d\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.253178 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.253144    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-svbfp"
Jul 25 01:12:37.365326 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.365292    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd88ee59-a0f7-4608-8c0a-77cc79d6e25b" containerID="360f9e036e922afe6a794ec68150fb082bb8b6b95ed11a72df7d2c8cf3f8faab" exitCode=0
Jul 25 01:12:37.365456 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.365341    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerDied","Data":"360f9e036e922afe6a794ec68150fb082bb8b6b95ed11a72df7d2c8cf3f8faab"}
Jul 25 01:12:37.379953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.379915    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-svbfp"]
Jul 25 01:12:37.384678 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:37.384653    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bd143fd_07cf_4981_981c_7f515ac8336f.slice/crio-4bc802140298176cb259bd7ad10113b8be36554053485a8519d2537f9dc13462 WatchSource:0}: Error finding container 4bc802140298176cb259bd7ad10113b8be36554053485a8519d2537f9dc13462: Status 404 returned error can't find the container with id 4bc802140298176cb259bd7ad10113b8be36554053485a8519d2537f9dc13462
Jul 25 01:12:37.649344 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:37.649315    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:37.649464 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:37.649420    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:12:37.649499 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:37.649470    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls podName:bb972c6d-e9ed-4b05-8817-1bed95eaae32 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:38.649456796 +0000 UTC m=+37.120413112 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls") pod "dns-default-b6rzf" (UID: "bb972c6d-e9ed-4b05-8817-1bed95eaae32") : secret "dns-default-metrics-tls" not found
Jul 25 01:12:38.069842 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.069812    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:12:38.069842 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.069837    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:12:38.070065 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.069821    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:12:38.072667 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.072635    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 25 01:12:38.072667 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.072646    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 25 01:12:38.073928 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.073792    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 25 01:12:38.073928 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.073801    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 25 01:12:38.073928 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.073842    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2rr8l\""
Jul 25 01:12:38.073928 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.073861    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2flvd\""
Jul 25 01:12:38.369429 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.369342    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-svbfp" event={"ID":"5bd143fd-07cf-4981-981c-7f515ac8336f","Type":"ContainerStarted","Data":"4bc802140298176cb259bd7ad10113b8be36554053485a8519d2537f9dc13462"}
Jul 25 01:12:38.372827 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.372801    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd88ee59-a0f7-4608-8c0a-77cc79d6e25b" containerID="d3c81bcbfd2423e2a9e5eb87b74664938aa4405096a3d7fb4bd9e89123e9bca1" exitCode=0
Jul 25 01:12:38.372947 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.372858    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerDied","Data":"d3c81bcbfd2423e2a9e5eb87b74664938aa4405096a3d7fb4bd9e89123e9bca1"}
Jul 25 01:12:38.657411 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:38.657170    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:38.657583 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:38.657343    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:12:38.657583 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:38.657529    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls podName:bb972c6d-e9ed-4b05-8817-1bed95eaae32 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:40.657499023 +0000 UTC m=+39.128455362 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls") pod "dns-default-b6rzf" (UID: "bb972c6d-e9ed-4b05-8817-1bed95eaae32") : secret "dns-default-metrics-tls" not found
Jul 25 01:12:39.377853 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:39.377817    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" event={"ID":"dd88ee59-a0f7-4608-8c0a-77cc79d6e25b","Type":"ContainerStarted","Data":"b40a4dc7e7d9bde87abd90da148e3c0a32bca15925b7df2a9cdcb19b79a66d03"}
Jul 25 01:12:39.403723 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:39.403603    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vl7nz" podStartSLOduration=8.244156965 podStartE2EDuration="37.403589126s" totalImagesPullingTime="29.159432161s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:39.401922153 +0000 UTC m=+37.872878489" watchObservedRunningTime="2026-07-25 01:12:39.403589126 +0000 UTC m=+37.874545465"
Jul 25 01:12:39.523376 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:39.523351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:39.585504 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:39.585470    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.131951 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.131919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.192710 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.192684    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.325535 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.325453    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.379784 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.379745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.381454 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.381431    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-svbfp" event={"ID":"5bd143fd-07cf-4981-981c-7f515ac8336f","Type":"ContainerStarted","Data":"3c851227c54c4f2ddcff317c0445826949c60ec350567b6f53668cdf7287a484"}
Jul 25 01:12:40.396826 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.396791    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-svbfp" podStartSLOduration=2.032779262 podStartE2EDuration="4.396777652s" totalImagesPullingTime="2.36399839s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:40.396613316 +0000 UTC m=+38.867569651" watchObservedRunningTime="2026-07-25 01:12:40.396777652 +0000 UTC m=+38.867733990"
Jul 25 01:12:40.591146 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.591064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.674165 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.674134    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:40.674280 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:40.674262    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:12:40.674334 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:40.674321    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls podName:bb972c6d-e9ed-4b05-8817-1bed95eaae32 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:44.674304176 +0000 UTC m=+43.145260491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls") pod "dns-default-b6rzf" (UID: "bb972c6d-e9ed-4b05-8817-1bed95eaae32") : secret "dns-default-metrics-tls" not found
Jul 25 01:12:40.691950 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.691924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.819824 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.819795    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq"]
Jul 25 01:12:40.820638 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.820609    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.842223 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.842164    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq"]
Jul 25 01:12:40.842333 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.842266    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq"
Jul 25 01:12:40.845277 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.845259    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Jul 25 01:12:40.845374 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.845304    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Jul 25 01:12:40.845549 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.845534    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-tw4xb\""
Jul 25 01:12:40.883298 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.883276    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.949289 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.949262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:40.977327 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:40.977307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6svp2\" (UniqueName: \"kubernetes.io/projected/b3144e4f-a7b6-454c-bec6-e90aa86bbce3-kube-api-access-6svp2\") pod \"migrator-6865c9967-fqvkq\" (UID: \"b3144e4f-a7b6-454c-bec6-e90aa86bbce3\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq"
Jul 25 01:12:41.068319 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.068293    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:41.078406 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.078382    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6svp2\" (UniqueName: \"kubernetes.io/projected/b3144e4f-a7b6-454c-bec6-e90aa86bbce3-kube-api-access-6svp2\") pod \"migrator-6865c9967-fqvkq\" (UID: \"b3144e4f-a7b6-454c-bec6-e90aa86bbce3\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq"
Jul 25 01:12:41.088334 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.088312    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6svp2\" (UniqueName: \"kubernetes.io/projected/b3144e4f-a7b6-454c-bec6-e90aa86bbce3-kube-api-access-6svp2\") pod \"migrator-6865c9967-fqvkq\" (UID: \"b3144e4f-a7b6-454c-bec6-e90aa86bbce3\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq"
Jul 25 01:12:41.122912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.122888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:41.150871 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.150853    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq"
Jul 25 01:12:41.203964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.203939    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-r9s4m"]
Jul 25 01:12:41.230321 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.230225    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r9s4m"]
Jul 25 01:12:41.230389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.230353    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.232783 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.232756    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Jul 25 01:12:41.232904 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.232799    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Jul 25 01:12:41.233479 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.233327    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mmjlg\""
Jul 25 01:12:41.234105 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.234075    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Jul 25 01:12:41.234105 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.234094    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Jul 25 01:12:41.250612 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.250561    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:41.303318 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.303295    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq"]
Jul 25 01:12:41.306414 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:41.306390    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3144e4f_a7b6_454c_bec6_e90aa86bbce3.slice/crio-3086d2642a210043bde86fc67be96fdb67f68149360cb233633c24803a2d3efa WatchSource:0}: Error finding container 3086d2642a210043bde86fc67be96fdb67f68149360cb233633c24803a2d3efa: Status 404 returned error can't find the container with id 3086d2642a210043bde86fc67be96fdb67f68149360cb233633c24803a2d3efa
Jul 25 01:12:41.313628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.313610    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:41.380854 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.380808    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n86x2\" (UniqueName: \"kubernetes.io/projected/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-api-access-n86x2\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.380854 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.380837    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.381179 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.380874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5577022-409f-4518-a9bd-77f7b0ffbcf0-crio-socket\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.381179 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.380888    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5577022-409f-4518-a9bd-77f7b0ffbcf0-data-volume\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.381179 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.380924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.384369 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.384340    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq" event={"ID":"b3144e4f-a7b6-454c-bec6-e90aa86bbce3","Type":"ContainerStarted","Data":"3086d2642a210043bde86fc67be96fdb67f68149360cb233633c24803a2d3efa"}
Jul 25 01:12:41.481330 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.481300    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.481452 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.481377    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5577022-409f-4518-a9bd-77f7b0ffbcf0-crio-socket\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.481452 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.481393    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5577022-409f-4518-a9bd-77f7b0ffbcf0-data-volume\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.481452 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.481409    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.481452 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:41.481439    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 01:12:41.481663 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:41.481492    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:41.481663 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:41.481521    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:41.981492577 +0000 UTC m=+40.452448913 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : configmap "kube-rbac-proxy" not found
Jul 25 01:12:41.481663 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.481605    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n86x2\" (UniqueName: \"kubernetes.io/projected/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-api-access-n86x2\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.481663 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:41.481633    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:41.98161901 +0000 UTC m=+40.452575337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:41.481663 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.481627    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/c5577022-409f-4518-a9bd-77f7b0ffbcf0-crio-socket\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.481879 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.481767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/c5577022-409f-4518-a9bd-77f7b0ffbcf0-data-volume\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.489782 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.489762    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n86x2\" (UniqueName: \"kubernetes.io/projected/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-api-access-n86x2\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.984774 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.984743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.984936 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:41.984803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:41.984936 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:41.984869    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 01:12:41.984936 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:41.984891    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:41.984936 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:41.984936    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:42.984915959 +0000 UTC m=+41.455872278 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : configmap "kube-rbac-proxy" not found
Jul 25 01:12:41.985198 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:41.985008    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:42.98498972 +0000 UTC m=+41.455946050 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:42.144377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:42.144346    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:42.200493 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:42.200467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:42.932195 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:42.932162    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:42.990322 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:42.990294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:42.993304 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:42.993284    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:42.993406 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:42.993339    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:42.993466 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:42.993417    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:42.993499 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:42.993475    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:44.993461101 +0000 UTC m=+43.464417418 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:42.993499 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:42.993418    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 01:12:42.993580 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:42.993546    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:44.9935316 +0000 UTC m=+43.464487918 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : configmap "kube-rbac-proxy" not found
Jul 25 01:12:43.401786 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:43.400715    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq" event={"ID":"b3144e4f-a7b6-454c-bec6-e90aa86bbce3","Type":"ContainerStarted","Data":"21e3e783c01abe28016af1e2d6e1536ec65c4508028d50a934d9f2883e966700"}
Jul 25 01:12:43.401786 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:43.401172    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq" event={"ID":"b3144e4f-a7b6-454c-bec6-e90aa86bbce3","Type":"ContainerStarted","Data":"70f788dec9b54028598a83c131d1b2a70a038bc553bc345dc2e68acffe284faa"}
Jul 25 01:12:43.422993 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:43.422931    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-fqvkq" podStartSLOduration=1.598242791 podStartE2EDuration="3.422918798s" totalImagesPullingTime="1.824676007s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:43.422118863 +0000 UTC m=+41.893075200" watchObservedRunningTime="2026-07-25 01:12:43.422918798 +0000 UTC m=+41.893875126"
Jul 25 01:12:43.532502 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:43.532432    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:43.822353 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:43.822264    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:44.200288 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:44.200260    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hljng_2f15f82d-6ba2-4275-a812-d1cd89ea9f69/dns-node-resolver/0.log"
Jul 25 01:12:44.706663 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:44.706625    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:44.706815 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:44.706786    2562 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Jul 25 01:12:44.706883 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:44.706864    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls podName:bb972c6d-e9ed-4b05-8817-1bed95eaae32 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:52.706847672 +0000 UTC m=+51.177803993 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls") pod "dns-default-b6rzf" (UID: "bb972c6d-e9ed-4b05-8817-1bed95eaae32") : secret "dns-default-metrics-tls" not found
Jul 25 01:12:45.009349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:45.009275    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:45.009349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:45.009331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:45.009543 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:45.009400    2562 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Jul 25 01:12:45.009543 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:45.009422    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:45.009543 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:45.009462    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:49.009447821 +0000 UTC m=+47.480404137 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : configmap "kube-rbac-proxy" not found
Jul 25 01:12:45.009543 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:45.009532    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:49.009517179 +0000 UTC m=+47.480473495 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:45.199974 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:45.199948    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-szrr5_52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8/node-ca/0.log"
Jul 25 01:12:45.800320 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:45.800294    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-svbfp_5bd143fd-07cf-4981-981c-7f515ac8336f/serve-healthcheck-canary/0.log"
Jul 25 01:12:46.202559 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:46.202526    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:46.202862 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:46.202849    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fqvkq_b3144e4f-a7b6-454c-bec6-e90aa86bbce3/migrator/0.log"
Jul 25 01:12:46.400677 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:46.400649    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fqvkq_b3144e4f-a7b6-454c-bec6-e90aa86bbce3/graceful-termination/0.log"
Jul 25 01:12:46.665050 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:46.665022    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:47.270449 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:47.270417    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:47.371570 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:47.371539    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:47.472308 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:47.472279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:47.613471 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:47.613398    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:47.673259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:47.673228    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:48.011496 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:48.011458    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:48.112323 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:48.112298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:48.367860 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:48.367784    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:49.040275 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:49.040239    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:49.040439 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:49.040331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:49.040439 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:49.040395    2562 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:49.040510 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:12:49.040476    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls podName:c5577022-409f-4518-a9bd-77f7b0ffbcf0 nodeName:}" failed. No retries permitted until 2026-07-25 01:12:57.040457182 +0000 UTC m=+55.511413504 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-r9s4m" (UID: "c5577022-409f-4518-a9bd-77f7b0ffbcf0") : secret "insights-runtime-extractor-tls" not found
Jul 25 01:12:49.040934 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:49.040915    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/c5577022-409f-4518-a9bd-77f7b0ffbcf0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:49.163504 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:49.163474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:49.367769 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:49.367694    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:49.764155 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:49.762807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:50.362247 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:50.362213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:50.882176 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:50.882145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:50.981947 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:50.981902    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:51.082959 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:51.082928    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:51.183034 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:51.182937    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:51.387708 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:51.387675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:51.560939 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:51.560858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:51.617906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:51.617874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:52.558472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:52.558436    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:52.758200 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:52.758169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:52.771939 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:52.771916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:52.774166 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:52.774140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bb972c6d-e9ed-4b05-8817-1bed95eaae32-metrics-tls\") pod \"dns-default-b6rzf\" (UID: \"bb972c6d-e9ed-4b05-8817-1bed95eaae32\") " pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:52.848689 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:52.848627    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:52.858914 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:52.858890    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:52.912166 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:52.912138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:52.985403 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:52.985376    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-b6rzf"]
Jul 25 01:12:52.988313 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:52.988288    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb972c6d_e9ed_4b05_8817_1bed95eaae32.slice/crio-877a5e9e07d35c6e957abe400dd7cc37a497167b33291fc9690465a8639050cc WatchSource:0}: Error finding container 877a5e9e07d35c6e957abe400dd7cc37a497167b33291fc9690465a8639050cc: Status 404 returned error can't find the container with id 877a5e9e07d35c6e957abe400dd7cc37a497167b33291fc9690465a8639050cc
Jul 25 01:12:53.164779 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:53.164745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:53.360691 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:53.360654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:53.423767 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:53.423693    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6rzf" event={"ID":"bb972c6d-e9ed-4b05-8817-1bed95eaae32","Type":"ContainerStarted","Data":"877a5e9e07d35c6e957abe400dd7cc37a497167b33291fc9690465a8639050cc"}
Jul 25 01:12:53.467927 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:53.467899    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:53.760036 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:53.759926    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:54.563223 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:54.563177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:54.768029 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:54.767988    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:54.957246 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:54.957218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:55.359642 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:55.359617    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:55.429581 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:55.429550    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6rzf" event={"ID":"bb972c6d-e9ed-4b05-8817-1bed95eaae32","Type":"ContainerStarted","Data":"808694a656859f0420581dc4ac614d2182d016f80b33df663fdbe3120ca0b686"}
Jul 25 01:12:55.429710 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:55.429602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-b6rzf" event={"ID":"bb972c6d-e9ed-4b05-8817-1bed95eaae32","Type":"ContainerStarted","Data":"2c4af5c9d21cfab5d2784f46225745764a642bec2f7e62fd0d3e090f08a00201"}
Jul 25 01:12:55.429710 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:55.429693    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:12:55.448943 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:55.448900    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-b6rzf" podStartSLOduration=17.306022388 podStartE2EDuration="19.448886628s" totalImagesPullingTime="2.14286424s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:12:55.44751893 +0000 UTC m=+53.918475268" watchObservedRunningTime="2026-07-25 01:12:55.448886628 +0000 UTC m=+53.919842979"
Jul 25 01:12:55.572696 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:55.572620    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:55.959974 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:55.959934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:56.363470 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:56.363367    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:56.768406 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:56.768366    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:57.102345 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.102250    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:57.104472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.104454    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/c5577022-409f-4518-a9bd-77f7b0ffbcf0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-r9s4m\" (UID: \"c5577022-409f-4518-a9bd-77f7b0ffbcf0\") " pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:57.140534 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.140505    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-r9s4m"
Jul 25 01:12:57.147869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.147842    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:57.210376 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.210343    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:57.258742 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.258691    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-r9s4m"]
Jul 25 01:12:57.261974 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:12:57.261949    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5577022_409f_4518_a9bd_77f7b0ffbcf0.slice/crio-c7b42e69b74fa06c4c67085c0ec4548cdef77a673c5082cf6467a2a9fdb451f8 WatchSource:0}: Error finding container c7b42e69b74fa06c4c67085c0ec4548cdef77a673c5082cf6467a2a9fdb451f8: Status 404 returned error can't find the container with id c7b42e69b74fa06c4c67085c0ec4548cdef77a673c5082cf6467a2a9fdb451f8
Jul 25 01:12:57.363572 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.363487    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:57.434900 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.434869    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r9s4m" event={"ID":"c5577022-409f-4518-a9bd-77f7b0ffbcf0","Type":"ContainerStarted","Data":"c7b42e69b74fa06c4c67085c0ec4548cdef77a673c5082cf6467a2a9fdb451f8"}
Jul 25 01:12:57.762211 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.762171    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:12:57.960296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:12:57.960204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:02.363936 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:02.363904    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-s7gf8"
Jul 25 01:13:03.454258 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:03.454228    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r9s4m" event={"ID":"c5577022-409f-4518-a9bd-77f7b0ffbcf0","Type":"ContainerStarted","Data":"54e5fc84b5afa69483cd9f6df10bb3d46d834f55c45062a01f47fa430b13e748"}
Jul 25 01:13:05.435045 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:05.435011    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-b6rzf"
Jul 25 01:13:06.036422 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.036377    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"]
Jul 25 01:13:06.039378 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.039356    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"]
Jul 25 01:13:06.039516 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.039478    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.042115 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.042089    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"
Jul 25 01:13:06.042370 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.042317    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Jul 25 01:13:06.042370 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.042322    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Jul 25 01:13:06.042535 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.042400    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Jul 25 01:13:06.042576 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.042530    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Jul 25 01:13:06.042637 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.042533    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Jul 25 01:13:06.043389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.043305    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Jul 25 01:13:06.043389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.043309    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Jul 25 01:13:06.044581 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.044565    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-h64p9\""
Jul 25 01:13:06.045033 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.045016    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Jul 25 01:13:06.049407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.049388    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"]
Jul 25 01:13:06.053613 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.053594    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"]
Jul 25 01:13:06.071291 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.071265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.071423 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.071329    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-ca\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.071423 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.071352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1416d88-788b-4c17-b639-c267e8a87f98-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.071423 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.071378    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2k6v\" (UniqueName: \"kubernetes.io/projected/61594f02-94b2-436c-9cf2-4a65a49e328c-kube-api-access-k2k6v\") pod \"managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj\" (UID: \"61594f02-94b2-436c-9cf2-4a65a49e328c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"
Jul 25 01:13:06.071578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.071465    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-hub\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.071578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.071499    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61594f02-94b2-436c-9cf2-4a65a49e328c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj\" (UID: \"61594f02-94b2-436c-9cf2-4a65a49e328c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"
Jul 25 01:13:06.071578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.071538    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.071578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.071565    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbkbp\" (UniqueName: \"kubernetes.io/projected/d1416d88-788b-4c17-b639-c267e8a87f98-kube-api-access-vbkbp\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.117384 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.117353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:06.172349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.172270    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-ca\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.172349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.172300    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1416d88-788b-4c17-b639-c267e8a87f98-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.172349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.172330    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2k6v\" (UniqueName: \"kubernetes.io/projected/61594f02-94b2-436c-9cf2-4a65a49e328c-kube-api-access-k2k6v\") pod \"managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj\" (UID: \"61594f02-94b2-436c-9cf2-4a65a49e328c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"
Jul 25 01:13:06.172657 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.172365    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-hub\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.172657 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.172410    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61594f02-94b2-436c-9cf2-4a65a49e328c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj\" (UID: \"61594f02-94b2-436c-9cf2-4a65a49e328c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"
Jul 25 01:13:06.172657 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.172460    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.172657 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.172478    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbkbp\" (UniqueName: \"kubernetes.io/projected/d1416d88-788b-4c17-b639-c267e8a87f98-kube-api-access-vbkbp\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.172657 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.172521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.173233 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.173206    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/d1416d88-788b-4c17-b639-c267e8a87f98-additional-service-ca\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.174954 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.174922    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-hub\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.174954 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.174938    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-ca\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.175123 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.175026    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.175480 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.175460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/d1416d88-788b-4c17-b639-c267e8a87f98-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.175520 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.175460    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/61594f02-94b2-436c-9cf2-4a65a49e328c-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj\" (UID: \"61594f02-94b2-436c-9cf2-4a65a49e328c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"
Jul 25 01:13:06.181357 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.181322    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2k6v\" (UniqueName: \"kubernetes.io/projected/61594f02-94b2-436c-9cf2-4a65a49e328c-kube-api-access-k2k6v\") pod \"managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj\" (UID: \"61594f02-94b2-436c-9cf2-4a65a49e328c\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"
Jul 25 01:13:06.182909 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.182889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbkbp\" (UniqueName: \"kubernetes.io/projected/d1416d88-788b-4c17-b639-c267e8a87f98-kube-api-access-vbkbp\") pod \"cluster-proxy-proxy-agent-79c6754678-8cbff\" (UID: \"d1416d88-788b-4c17-b639-c267e8a87f98\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.215801 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.215773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:06.355588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.355555    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"
Jul 25 01:13:06.360777 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.360760    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"
Jul 25 01:13:06.466421 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.466394    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r9s4m" event={"ID":"c5577022-409f-4518-a9bd-77f7b0ffbcf0","Type":"ContainerStarted","Data":"d32dc06b5abefca431d07e11594a28a103fca22c909a5b2c7c3d59e0a3f2a147"}
Jul 25 01:13:06.516817 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.516778    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff"]
Jul 25 01:13:06.521119 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:13:06.521081    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd1416d88_788b_4c17_b639_c267e8a87f98.slice/crio-529ce924990a72080cb799e268264c16e148b9f3cceed619cd025d3dcf280bbe WatchSource:0}: Error finding container 529ce924990a72080cb799e268264c16e148b9f3cceed619cd025d3dcf280bbe: Status 404 returned error can't find the container with id 529ce924990a72080cb799e268264c16e148b9f3cceed619cd025d3dcf280bbe
Jul 25 01:13:06.530422 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.530375    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj"]
Jul 25 01:13:06.535309 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:13:06.535282    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod61594f02_94b2_436c_9cf2_4a65a49e328c.slice/crio-c5329921840571c330d052be02ff0922459fc3adaf5ee48e15121e3ab7bc5d4c WatchSource:0}: Error finding container c5329921840571c330d052be02ff0922459fc3adaf5ee48e15121e3ab7bc5d4c: Status 404 returned error can't find the container with id c5329921840571c330d052be02ff0922459fc3adaf5ee48e15121e3ab7bc5d4c
Jul 25 01:13:06.777352 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.777274    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:13:06.777631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.777356    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:13:06.780202 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.780180    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Jul 25 01:13:06.780351 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.780244    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Jul 25 01:13:06.790862 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.790841    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/725501d7-58c0-4b02-9e3f-8e931b0d0dfc-metrics-certs\") pod \"network-metrics-daemon-w6nf6\" (UID: \"725501d7-58c0-4b02-9e3f-8e931b0d0dfc\") " pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:13:06.790992 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.790843    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e1fb37b0-fa56-4167-873a-c72bb0947ebd-original-pull-secret\") pod \"global-pull-secret-syncer-c46qr\" (UID: \"e1fb37b0-fa56-4167-873a-c72bb0947ebd\") " pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:13:06.878586 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.878550    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:13:06.881501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.881481    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Jul 25 01:13:06.891439 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.891414    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Jul 25 01:13:06.892563 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.892546    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-2flvd\""
Jul 25 01:13:06.895717 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.895695    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-c46qr"
Jul 25 01:13:06.900452 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.900435    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-w6nf6"
Jul 25 01:13:06.901849 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:06.901826    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx7tm\" (UniqueName: \"kubernetes.io/projected/500981ce-a9b7-4efb-8d4b-8f0ca9ceec63-kube-api-access-bx7tm\") pod \"network-check-target-5fqmf\" (UID: \"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63\") " pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:13:07.033951 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.033880    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-c46qr"]
Jul 25 01:13:07.037278 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:13:07.037129    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1fb37b0_fa56_4167_873a_c72bb0947ebd.slice/crio-4958b22e4a538e15d9d507dd3edc542f3640e45ed517c4245ce1966ba13d43ed WatchSource:0}: Error finding container 4958b22e4a538e15d9d507dd3edc542f3640e45ed517c4245ce1966ba13d43ed: Status 404 returned error can't find the container with id 4958b22e4a538e15d9d507dd3edc542f3640e45ed517c4245ce1966ba13d43ed
Jul 25 01:13:07.054516 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.054490    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-w6nf6"]
Jul 25 01:13:07.057396 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:13:07.057372    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725501d7_58c0_4b02_9e3f_8e931b0d0dfc.slice/crio-85a3ca540d33190ad84f7c52f0803263ffff380e70226498fc15f88517ce980d WatchSource:0}: Error finding container 85a3ca540d33190ad84f7c52f0803263ffff380e70226498fc15f88517ce980d: Status 404 returned error can't find the container with id 85a3ca540d33190ad84f7c52f0803263ffff380e70226498fc15f88517ce980d
Jul 25 01:13:07.184405 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.184376    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2rr8l\""
Jul 25 01:13:07.192835 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.192807    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:13:07.334620 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.334556    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-5fqmf"]
Jul 25 01:13:07.337768 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:13:07.337744    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod500981ce_a9b7_4efb_8d4b_8f0ca9ceec63.slice/crio-651f81a1d3ca5b81685886c90a161b35882f48c441a1e0b477b236d33cb772da WatchSource:0}: Error finding container 651f81a1d3ca5b81685886c90a161b35882f48c441a1e0b477b236d33cb772da: Status 404 returned error can't find the container with id 651f81a1d3ca5b81685886c90a161b35882f48c441a1e0b477b236d33cb772da
Jul 25 01:13:07.476555 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.476478    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c46qr" event={"ID":"e1fb37b0-fa56-4167-873a-c72bb0947ebd","Type":"ContainerStarted","Data":"4958b22e4a538e15d9d507dd3edc542f3640e45ed517c4245ce1966ba13d43ed"}
Jul 25 01:13:07.479775 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.479696    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5fqmf" event={"ID":"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63","Type":"ContainerStarted","Data":"651f81a1d3ca5b81685886c90a161b35882f48c441a1e0b477b236d33cb772da"}
Jul 25 01:13:07.481519 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.481407    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w6nf6" event={"ID":"725501d7-58c0-4b02-9e3f-8e931b0d0dfc","Type":"ContainerStarted","Data":"85a3ca540d33190ad84f7c52f0803263ffff380e70226498fc15f88517ce980d"}
Jul 25 01:13:07.485564 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.485496    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff" event={"ID":"d1416d88-788b-4c17-b639-c267e8a87f98","Type":"ContainerStarted","Data":"529ce924990a72080cb799e268264c16e148b9f3cceed619cd025d3dcf280bbe"}
Jul 25 01:13:07.487080 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:07.487027    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj" event={"ID":"61594f02-94b2-436c-9cf2-4a65a49e328c","Type":"ContainerStarted","Data":"c5329921840571c330d052be02ff0922459fc3adaf5ee48e15121e3ab7bc5d4c"}
Jul 25 01:13:10.919133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:10.919097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:11.019414 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:11.019345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.075325 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.075294    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.175384 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.175341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.272191 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.272157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.336185 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.336112    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.401747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.401708    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.485731 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.485697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.546093 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.546064    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.646589 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.646556    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.703640 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.703603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.935623 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.935542    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:12.997256 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:12.997211    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:13.062264 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:13.062233    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:13.163645 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:13.163615    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:13.342021 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:13.341929    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:13.572392 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:13.572355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:13.627989 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:13.627935    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:13.768928 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:13.768897    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:13.868929 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:13.868888    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:13.970009 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:13.969912    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:14.200972 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:14.200934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:14.301392 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:14.301303    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:15.089095 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:15.089004    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:15.141213 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:15.141145    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:15.566702 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:15.566664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:15.625997 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:15.625950    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:16.522915 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.522883    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-c46qr" event={"ID":"e1fb37b0-fa56-4167-873a-c72bb0947ebd","Type":"ContainerStarted","Data":"16eb38ab77d5a11b7c31a7d681a39f655784ec1a13da787c82f2aa789177b444"}
Jul 25 01:13:16.525505 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.524872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-5fqmf" event={"ID":"500981ce-a9b7-4efb-8d4b-8f0ca9ceec63","Type":"ContainerStarted","Data":"73f303d0dff28091fefa538f9a324834550dfb08de2aacc1902488ad70a02968"}
Jul 25 01:13:16.525505 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.525435    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:13:16.527731 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.527697    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w6nf6" event={"ID":"725501d7-58c0-4b02-9e3f-8e931b0d0dfc","Type":"ContainerStarted","Data":"c0f902b4ad17accb304a05ee00bc4874ce988bfcacb3c8a09def340baeeb2063"}
Jul 25 01:13:16.527847 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.527748    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-w6nf6" event={"ID":"725501d7-58c0-4b02-9e3f-8e931b0d0dfc","Type":"ContainerStarted","Data":"638bf54c3e71382fb3732ee1721af44115bb0f8d781aa7c7fb1dfad9c66242f7"}
Jul 25 01:13:16.529872 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.529852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff" event={"ID":"d1416d88-788b-4c17-b639-c267e8a87f98","Type":"ContainerStarted","Data":"cc861d34f2c434a2b348acb39964f536ee233a2bfcbfa83d66d099504d4888a4"}
Jul 25 01:13:16.529969 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.529878    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff" event={"ID":"d1416d88-788b-4c17-b639-c267e8a87f98","Type":"ContainerStarted","Data":"d888e7c50298994754745b9fcecbeeb9309eeacb391059cd1181db5c34298a83"}
Jul 25 01:13:16.529969 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.529890    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff" event={"ID":"d1416d88-788b-4c17-b639-c267e8a87f98","Type":"ContainerStarted","Data":"04e8065b2d5e1fbba11c5825f5c9c4e6bbefea4133e983e0ee833cab85146e95"}
Jul 25 01:13:16.531535 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.531516    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-r9s4m" event={"ID":"c5577022-409f-4518-a9bd-77f7b0ffbcf0","Type":"ContainerStarted","Data":"601acda0976eb164f31b9893c36f0e303ae6432c99a9954f1e160e4aeda664ac"}
Jul 25 01:13:16.532751 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.532732    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj" event={"ID":"61594f02-94b2-436c-9cf2-4a65a49e328c","Type":"ContainerStarted","Data":"53daf30227c6261f38339c5a97531f52b7fd91caff45f3341c689610d9a76d3c"}
Jul 25 01:13:16.539779 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.539737    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-c46qr" podStartSLOduration=65.748982196 podStartE2EDuration="1m14.539724605s" totalImagesPullingTime="8.790742409s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:16.539092002 +0000 UTC m=+75.010048341" watchObservedRunningTime="2026-07-25 01:13:16.539724605 +0000 UTC m=+75.010680921"
Jul 25 01:13:16.557767 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.557730    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-w6nf6" podStartSLOduration=65.830706891 podStartE2EDuration="1m14.557717719s" totalImagesPullingTime="8.727010828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:16.557615806 +0000 UTC m=+75.028572144" watchObservedRunningTime="2026-07-25 01:13:16.557717719 +0000 UTC m=+75.028674056"
Jul 25 01:13:16.575647 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.575609    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-r9s4m" podStartSLOduration=17.212409042 podStartE2EDuration="35.575598043s" totalImagesPullingTime="18.363189001s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 01:12:41 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:16.575107113 +0000 UTC m=+75.046063463" watchObservedRunningTime="2026-07-25 01:13:16.575598043 +0000 UTC m=+75.046554380"
Jul 25 01:13:16.594706 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.594666    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-84cc4b5dbd-7mtbj" podStartSLOduration=1.340337929 podStartE2EDuration="10.594655028s" totalImagesPullingTime="9.254317099s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:16.593603035 +0000 UTC m=+75.064559372" watchObservedRunningTime="2026-07-25 01:13:16.594655028 +0000 UTC m=+75.065611343"
Jul 25 01:13:16.611607 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.611572    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-5fqmf" podStartSLOduration=66.122222544 podStartE2EDuration="1m14.61156126s" totalImagesPullingTime="8.489338716s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:12:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:16.610929101 +0000 UTC m=+75.081885440" watchObservedRunningTime="2026-07-25 01:13:16.61156126 +0000 UTC m=+75.082517598"
Jul 25 01:13:16.633212 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.633177    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-79c6754678-8cbff" podStartSLOduration=1.299012809 podStartE2EDuration="10.633163316s" totalImagesPullingTime="9.334150507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:13:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:16.632027277 +0000 UTC m=+75.102983616" watchObservedRunningTime="2026-07-25 01:13:16.633163316 +0000 UTC m=+75.104119680"
Jul 25 01:13:16.663690 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.663645    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:16.767690 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:16.767661    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:17.257858 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:17.257801    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:17.329705 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:17.329676    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:17.615673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:17.615571    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:21.050486 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:21.050421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:23.437623 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.437587    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:23.591030 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.590877    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:23.686926 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.686899    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-x65db"]
Jul 25 01:13:23.691251 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.691204    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.694047 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.694028    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Jul 25 01:13:23.694154 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.694133    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Jul 25 01:13:23.694228 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.694153    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Jul 25 01:13:23.695062 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.695045    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Jul 25 01:13:23.695208 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.695125    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Jul 25 01:13:23.695302 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.695225    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Jul 25 01:13:23.695302 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.695250    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-z8cbs\""
Jul 25 01:13:23.702560 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.702543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:23.799896 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.799865    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/493c913a-3a85-4898-9045-967a926db33a-node-exporter-accelerators-collector-config\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.799896 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.799901    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/493c913a-3a85-4898-9045-967a926db33a-node-exporter-textfile\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.800131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.799929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-root\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.800131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.799954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-sys\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.800131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.800002    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/493c913a-3a85-4898-9045-967a926db33a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.800131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.800032    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vsz9\" (UniqueName: \"kubernetes.io/projected/493c913a-3a85-4898-9045-967a926db33a-kube-api-access-9vsz9\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.800131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.800048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/493c913a-3a85-4898-9045-967a926db33a-metrics-client-ca\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.800131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.800080    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-node-exporter-wtmp\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.800349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.800130    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/493c913a-3a85-4898-9045-967a926db33a-node-exporter-tls\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.900861 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.900826    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/493c913a-3a85-4898-9045-967a926db33a-node-exporter-tls\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901086 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.900867    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/493c913a-3a85-4898-9045-967a926db33a-node-exporter-accelerators-collector-config\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901086 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.900894    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/493c913a-3a85-4898-9045-967a926db33a-node-exporter-textfile\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901086 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.900915    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-root\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901086 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.900930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-sys\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901086 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.900947    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/493c913a-3a85-4898-9045-967a926db33a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901086 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.900974    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vsz9\" (UniqueName: \"kubernetes.io/projected/493c913a-3a85-4898-9045-967a926db33a-kube-api-access-9vsz9\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901086 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.901065    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-sys\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901441 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.901086    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-root\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901441 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.901117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/493c913a-3a85-4898-9045-967a926db33a-metrics-client-ca\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901441 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.901155    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-node-exporter-wtmp\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901441 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.901289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/493c913a-3a85-4898-9045-967a926db33a-node-exporter-textfile\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901441 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.901298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/493c913a-3a85-4898-9045-967a926db33a-node-exporter-wtmp\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901625 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.901601    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/493c913a-3a85-4898-9045-967a926db33a-node-exporter-accelerators-collector-config\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.901671 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.901655    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/493c913a-3a85-4898-9045-967a926db33a-metrics-client-ca\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.903229 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.903210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/493c913a-3a85-4898-9045-967a926db33a-node-exporter-tls\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.903329 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.903258    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/493c913a-3a85-4898-9045-967a926db33a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:23.904889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.904863    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:23.913423 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:23.913399    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vsz9\" (UniqueName: \"kubernetes.io/projected/493c913a-3a85-4898-9045-967a926db33a-kube-api-access-9vsz9\") pod \"node-exporter-x65db\" (UID: \"493c913a-3a85-4898-9045-967a926db33a\") " pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:24.000742 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.000659    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-x65db"
Jul 25 01:13:24.001279 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.001251    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:24.012195 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:13:24.012167    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod493c913a_3a85_4898_9045_967a926db33a.slice/crio-4fff52951d47ad0f41db74188c7592fe2d06f482599ed6e840dfe7ce2f379c85 WatchSource:0}: Error finding container 4fff52951d47ad0f41db74188c7592fe2d06f482599ed6e840dfe7ce2f379c85: Status 404 returned error can't find the container with id 4fff52951d47ad0f41db74188c7592fe2d06f482599ed6e840dfe7ce2f379c85
Jul 25 01:13:24.101305 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.101278    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:24.220662 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.220627    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:24.319285 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.319191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:24.419385 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.419351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:24.559671 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.559637    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x65db" event={"ID":"493c913a-3a85-4898-9045-967a926db33a","Type":"ContainerStarted","Data":"4fff52951d47ad0f41db74188c7592fe2d06f482599ed6e840dfe7ce2f379c85"}
Jul 25 01:13:24.663409 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.663377    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:24.783856 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.783826    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:24.985248 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:24.985218    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:25.565027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:25.564993    2562 generic.go:356] "Generic (PLEG): container finished" podID="493c913a-3a85-4898-9045-967a926db33a" containerID="cc4989421b2279082f51837a312950d2508d4b87d204d2aaa624d92b9a567a71" exitCode=0
Jul 25 01:13:25.565395 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:25.565056    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x65db" event={"ID":"493c913a-3a85-4898-9045-967a926db33a","Type":"ContainerDied","Data":"cc4989421b2279082f51837a312950d2508d4b87d204d2aaa624d92b9a567a71"}
Jul 25 01:13:25.712638 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:25.712604    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:25.818272 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:25.818185    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:26.570025 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:26.569966    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x65db" event={"ID":"493c913a-3a85-4898-9045-967a926db33a","Type":"ContainerStarted","Data":"ab50b240700c0e718adfe6fe7be7dfc412693cdd0c48ff58ed2eae62a62e9d61"}
Jul 25 01:13:26.570418 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:26.570050    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-x65db" event={"ID":"493c913a-3a85-4898-9045-967a926db33a","Type":"ContainerStarted","Data":"177f4298b0be84ec3cb5bcfb26706e77c74493ec69587b3a5c3ba51deb93a92a"}
Jul 25 01:13:26.599514 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:26.599465    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-x65db" podStartSLOduration=2.721098649 podStartE2EDuration="3.599448819s" totalImagesPullingTime="815.207231ms" totalInitContainerRuntime="63.142939ms" isStatefulPod=true podCreationTimestamp="2026-07-25 01:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:26.59727998 +0000 UTC m=+85.068236309" watchObservedRunningTime="2026-07-25 01:13:26.599448819 +0000 UTC m=+85.070405159"
Jul 25 01:13:26.778722 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:26.778688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:26.890524 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:26.890490    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:26.987526 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:26.987497    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:27.090098 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:27.090056    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:28.424318 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.424282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:28.430167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.430143    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"]
Jul 25 01:13:28.435615 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.435600    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:28.438181 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.438162    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
Jul 25 01:13:28.438282 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.438163    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-st5xd\""
Jul 25 01:13:28.445175 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.445155    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"]
Jul 25 01:13:28.485365 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.485332    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:28.536092 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.536060    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd9pz\" (UniqueName: \"kubernetes.io/projected/431a96c2-8ba7-4163-9b06-cf3e14872286-kube-api-access-xd9pz\") pod \"monitoring-plugin-68b7fc9574-j2wx8\" (UID: \"431a96c2-8ba7-4163-9b06-cf3e14872286\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:28.536238 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.536125    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/431a96c2-8ba7-4163-9b06-cf3e14872286-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-j2wx8\" (UID: \"431a96c2-8ba7-4163-9b06-cf3e14872286\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:28.637125 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.637091    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/431a96c2-8ba7-4163-9b06-cf3e14872286-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-j2wx8\" (UID: \"431a96c2-8ba7-4163-9b06-cf3e14872286\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:28.637299 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.637135    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xd9pz\" (UniqueName: \"kubernetes.io/projected/431a96c2-8ba7-4163-9b06-cf3e14872286-kube-api-access-xd9pz\") pod \"monitoring-plugin-68b7fc9574-j2wx8\" (UID: \"431a96c2-8ba7-4163-9b06-cf3e14872286\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:28.637299 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:13:28.637237    2562 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
Jul 25 01:13:28.637375 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:13:28.637304    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/431a96c2-8ba7-4163-9b06-cf3e14872286-monitoring-plugin-cert podName:431a96c2-8ba7-4163-9b06-cf3e14872286 nodeName:}" failed. No retries permitted until 2026-07-25 01:13:29.137284126 +0000 UTC m=+87.608240453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/431a96c2-8ba7-4163-9b06-cf3e14872286-monitoring-plugin-cert") pod "monitoring-plugin-68b7fc9574-j2wx8" (UID: "431a96c2-8ba7-4163-9b06-cf3e14872286") : secret "monitoring-plugin-cert" not found
Jul 25 01:13:28.648028 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.647971    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd9pz\" (UniqueName: \"kubernetes.io/projected/431a96c2-8ba7-4163-9b06-cf3e14872286-kube-api-access-xd9pz\") pod \"monitoring-plugin-68b7fc9574-j2wx8\" (UID: \"431a96c2-8ba7-4163-9b06-cf3e14872286\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:28.715148 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:28.715040    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:29.140227 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.140195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/431a96c2-8ba7-4163-9b06-cf3e14872286-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-j2wx8\" (UID: \"431a96c2-8ba7-4163-9b06-cf3e14872286\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:29.142510 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.142489    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/431a96c2-8ba7-4163-9b06-cf3e14872286-monitoring-plugin-cert\") pod \"monitoring-plugin-68b7fc9574-j2wx8\" (UID: \"431a96c2-8ba7-4163-9b06-cf3e14872286\") " pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:29.343801 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.343774    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:29.470323 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.470294    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"]
Jul 25 01:13:29.473261 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:13:29.473235    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod431a96c2_8ba7_4163_9b06_cf3e14872286.slice/crio-5762e07f41163cc0ff6b833d9f299aa00048bf5bc30408dc2664ba87f85daba6 WatchSource:0}: Error finding container 5762e07f41163cc0ff6b833d9f299aa00048bf5bc30408dc2664ba87f85daba6: Status 404 returned error can't find the container with id 5762e07f41163cc0ff6b833d9f299aa00048bf5bc30408dc2664ba87f85daba6
Jul 25 01:13:29.581041 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.581009    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8" event={"ID":"431a96c2-8ba7-4163-9b06-cf3e14872286","Type":"ContainerStarted","Data":"5762e07f41163cc0ff6b833d9f299aa00048bf5bc30408dc2664ba87f85daba6"}
Jul 25 01:13:29.879857 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.879812    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:29.984044 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.984006    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:29.989526 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.989503    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:13:29.995309 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.995295    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:29.998626 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.998609    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ppv7x\""
Jul 25 01:13:29.998763 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:29.998749    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 25 01:13:30.002085 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002063    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 25 01:13:30.002195 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002085    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 25 01:13:30.002195 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002171    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 25 01:13:30.002289 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002210    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 25 01:13:30.002338 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002291    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 25 01:13:30.002502 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002481    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 25 01:13:30.002611 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002511    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 25 01:13:30.002611 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002519    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 25 01:13:30.002611 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.002586    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 25 01:13:30.003454 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.003439    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 25 01:13:30.003454 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.003446    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 25 01:13:30.003605 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.003458    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 25 01:13:30.003605 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.003439    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 25 01:13:30.003605 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.003514    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4vcvtk199ss30\""
Jul 25 01:13:30.004612 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.004593    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 25 01:13:30.017293 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.017274    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:13:30.047703 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fj5c8\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-kube-api-access-fj5c8\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.047816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047716    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.047816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047745    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.047816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047765    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.047816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047800    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047823    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047843    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047867    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047939    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.047989    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048023    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048050    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048131    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048175    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048209    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048265    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048283    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048532 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048305    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048532 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048320    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.048532 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.048334    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.144295 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.144213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:30.149167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149145    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.149312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149174    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.149312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149217    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.149312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149243    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.149475 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149463    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.149540 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.149540 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149521    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.149647 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149553    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.149647 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149589    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.150012 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149805    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.150012 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.149889    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.150540 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150289    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150675    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150712    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150736    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fj5c8\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-kube-api-access-fj5c8\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150843    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150880    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.150970    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.151632    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.152477 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.152170    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.153144 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.152582    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config-out\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.153144 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.152762    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.153144 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.153080    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.153307 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.153187    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.153359 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.153309    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.153878 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.153851    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.154043 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.154022    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.154135 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.154093    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.154752 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.154729    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.154899 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.154738    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.155486 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.155428    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.156621 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.155639    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.156621 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.155951    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.156761 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.156684    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.156943 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.156918    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-web-config\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.161922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.161904    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fj5c8\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-kube-api-access-fj5c8\") pod \"prometheus-k8s-0\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.305137 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.305102    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:30.437006 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.436922    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:13:30.439949 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:13:30.439916    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd7e9f3a_425e_4e05_9276_d58bc415bf18.slice/crio-ccea264af146ad0c8e668d2ca0c7458bd8088e1d1ee7c56643a4e976ffc8ab50 WatchSource:0}: Error finding container ccea264af146ad0c8e668d2ca0c7458bd8088e1d1ee7c56643a4e976ffc8ab50: Status 404 returned error can't find the container with id ccea264af146ad0c8e668d2ca0c7458bd8088e1d1ee7c56643a4e976ffc8ab50
Jul 25 01:13:30.591126 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:30.591095    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerStarted","Data":"ccea264af146ad0c8e668d2ca0c7458bd8088e1d1ee7c56643a4e976ffc8ab50"}
Jul 25 01:13:31.596020 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:31.595963    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2" exitCode=0
Jul 25 01:13:31.596352 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:31.596044    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerDied","Data":"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"}
Jul 25 01:13:31.597530 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:31.597508    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8" event={"ID":"431a96c2-8ba7-4163-9b06-cf3e14872286","Type":"ContainerStarted","Data":"122dce9d898e0b38c6585ddd5f7744dec417f6e8cc2280a349ec3154115d4b30"}
Jul 25 01:13:31.597786 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:31.597717    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:31.602599 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:31.602582    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8"
Jul 25 01:13:31.668187 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:31.668131    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-68b7fc9574-j2wx8" podStartSLOduration=2.2877903330000002 podStartE2EDuration="3.668116119s" totalImagesPullingTime="1.380325786s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:13:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:31.66688653 +0000 UTC m=+90.137842868" watchObservedRunningTime="2026-07-25 01:13:31.668116119 +0000 UTC m=+90.139072457"
Jul 25 01:13:31.845558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:31.845467    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:31.945057 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:31.945018    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:32.046353 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:32.046322    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:34.132831 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:34.132794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:34.193854 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:34.193813    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:35.615662 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:35.615579    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerStarted","Data":"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"}
Jul 25 01:13:35.615662 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:35.615648    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerStarted","Data":"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"}
Jul 25 01:13:37.575462 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:37.575433    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:37.625354 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:37.625293    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerStarted","Data":"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"}
Jul 25 01:13:37.625468 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:37.625417    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerStarted","Data":"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"}
Jul 25 01:13:37.676064 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:37.676032    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:38.631837 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:38.631799    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerStarted","Data":"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"}
Jul 25 01:13:38.632228 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:38.631878    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerStarted","Data":"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"}
Jul 25 01:13:38.660886 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:38.660823    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.138664178 podStartE2EDuration="9.660802652s" totalImagesPullingTime="6.503913355s" totalInitContainerRuntime="18.225119ms" isStatefulPod=false podCreationTimestamp="2026-07-25 01:13:29 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:13:38.658336252 +0000 UTC m=+97.129292592" watchObservedRunningTime="2026-07-25 01:13:38.660802652 +0000 UTC m=+97.131758993"
Jul 25 01:13:38.856896 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:38.856858    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:38.910560 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:38.910484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:40.181909 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:40.181868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:40.287352 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:40.287284    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:40.306114 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:40.306097    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:13:42.145087 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:42.145055    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:42.200525 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:42.200492    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:44.031416 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:44.031380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:44.132073 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:44.132039    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:44.186809 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:44.186778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:46.228624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:46.228590    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-svbfp_5bd143fd-07cf-4981-981c-7f515ac8336f/serve-healthcheck-canary/0.log"
Jul 25 01:13:48.541393 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:48.541361    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-5fqmf"
Jul 25 01:13:57.149619 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:57.149583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:13:57.208731 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:13:57.208688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:04.510817 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:04.510774    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:04.611853 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:04.611827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:04.711660 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:04.711629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:30.305381 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:30.305347    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:30.326041 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:30.326016    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:30.825426 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:30.825401    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:43.029398 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:43.029353    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:43.278771 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:43.278733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:48.386819 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.386781    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:14:48.387273 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.387118    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="prometheus" containerID="cri-o://c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498" gracePeriod=600
Jul 25 01:14:48.387273 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.387134    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy-thanos" containerID="cri-o://48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6" gracePeriod=600
Jul 25 01:14:48.387273 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.387160    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy" containerID="cri-o://0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51" gracePeriod=600
Jul 25 01:14:48.387273 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.387170    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="config-reloader" containerID="cri-o://c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231" gracePeriod=600
Jul 25 01:14:48.387273 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.387128    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy-web" containerID="cri-o://e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628" gracePeriod=600
Jul 25 01:14:48.387273 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.387239    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="thanos-sidecar" containerID="cri-o://b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea" gracePeriod=600
Jul 25 01:14:48.621172 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.621150    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:48.709971 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.709910    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-web-config\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.709971 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.709941    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-kube-rbac-proxy\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.709971 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.709959    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-tls-assets\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.709989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-db\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710022    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-metrics-client-certs\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710062    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-tls\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710090    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710124    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710157    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-kubelet-serving-ca-bundle\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710189    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-trusted-ca-bundle\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710218    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-1\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710255    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-serving-certs-ca-bundle\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710279    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710304    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-metrics-client-ca\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710346    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-2\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710394    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-0\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710418    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-grpc-tls\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710450    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config-out\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710492    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fj5c8\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-kube-api-access-fj5c8\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710519    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-thanos-prometheus-http-client-file\") pod \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\" (UID: \"dd7e9f3a-425e-4e05-9276-d58bc415bf18\") "
Jul 25 01:14:48.710822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.710801    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:48.711869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.711574    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:48.711869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.711582    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:48.711869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.711834    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:14:48.712092 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.711924    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:48.712697 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.712616    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:48.712951 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.712926    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:48.713228 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.713198    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:14:48.713314 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.713281    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:14:48.713314 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.713289    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.713433 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.713319    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.713590 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.713560    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.714021 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.713959    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.714804 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.714782    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config-out" (OuterVolumeSpecName: "config-out") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:14:48.714804 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.714790    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.714938 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.714915    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.715089 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.715066    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-kube-api-access-fj5c8" (OuterVolumeSpecName: "kube-api-access-fj5c8") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "kube-api-access-fj5c8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:14:48.715167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.715108    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.715618 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.715600    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config" (OuterVolumeSpecName: "config") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.723296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.723275    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-web-config" (OuterVolumeSpecName: "web-config") pod "dd7e9f3a-425e-4e05-9276-d58bc415bf18" (UID: "dd7e9f3a-425e-4e05-9276-d58bc415bf18"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:14:48.811167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811145    2562 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811164    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-metrics-client-ca\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811175    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811185    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811195    2562 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-grpc-tls\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811204    2562 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-config-out\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811212    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fj5c8\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-kube-api-access-fj5c8\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811221    2562 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-thanos-prometheus-http-client-file\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811231    2562 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-web-config\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811242    2562 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-kube-rbac-proxy\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811250    2562 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/dd7e9f3a-425e-4e05-9276-d58bc415bf18-tls-assets\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811258    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-db\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811266    2562 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-metrics-client-certs\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811274    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-tls\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811283    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811650 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811291    2562 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/dd7e9f3a-425e-4e05-9276-d58bc415bf18-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811650 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811301    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811650 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811309    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-trusted-ca-bundle\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811650 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811317    2562 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.811650 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.811325    2562 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/dd7e9f3a-425e-4e05-9276-d58bc415bf18-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:14:48.875828 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875800    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6" exitCode=0
Jul 25 01:14:48.875828 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875824    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51" exitCode=0
Jul 25 01:14:48.875828 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875832    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628" exitCode=0
Jul 25 01:14:48.875828 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875837    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea" exitCode=0
Jul 25 01:14:48.876072 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875843    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231" exitCode=0
Jul 25 01:14:48.876072 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875849    2562 generic.go:356] "Generic (PLEG): container finished" podID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498" exitCode=0
Jul 25 01:14:48.876072 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875877    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:48.876072 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875876    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerDied","Data":"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"}
Jul 25 01:14:48.876072 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.875959    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerDied","Data":"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"}
Jul 25 01:14:48.876072 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.876012    2562 scope.go:117] "RemoveContainer" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"
Jul 25 01:14:48.876072 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.876018    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerDied","Data":"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"}
Jul 25 01:14:48.876326 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.876076    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerDied","Data":"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"}
Jul 25 01:14:48.876326 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.876117    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerDied","Data":"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"}
Jul 25 01:14:48.876326 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.876159    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerDied","Data":"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"}
Jul 25 01:14:48.876326 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.876201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"dd7e9f3a-425e-4e05-9276-d58bc415bf18","Type":"ContainerDied","Data":"ccea264af146ad0c8e668d2ca0c7458bd8088e1d1ee7c56643a4e976ffc8ab50"}
Jul 25 01:14:48.883923 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.883786    2562 scope.go:117] "RemoveContainer" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"
Jul 25 01:14:48.890346 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.890327    2562 scope.go:117] "RemoveContainer" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"
Jul 25 01:14:48.896557 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.896542    2562 scope.go:117] "RemoveContainer" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"
Jul 25 01:14:48.900123 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.900102    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:14:48.903749 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.903731    2562 scope.go:117] "RemoveContainer" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"
Jul 25 01:14:48.904355 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.904337    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:14:48.911241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.911222    2562 scope.go:117] "RemoveContainer" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"
Jul 25 01:14:48.917711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.917697    2562 scope.go:117] "RemoveContainer" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"
Jul 25 01:14:48.923557 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.923542    2562 scope.go:117] "RemoveContainer" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"
Jul 25 01:14:48.923803 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:14:48.923783    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": container with ID starting with 48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6 not found: ID does not exist" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"
Jul 25 01:14:48.923875 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.923812    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"} err="failed to get container status \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": rpc error: code = NotFound desc = could not find container \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": container with ID starting with 48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6 not found: ID does not exist"
Jul 25 01:14:48.923875 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.923852    2562 scope.go:117] "RemoveContainer" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"
Jul 25 01:14:48.924241 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:14:48.924224    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": container with ID starting with 0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51 not found: ID does not exist" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"
Jul 25 01:14:48.924300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.924246    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"} err="failed to get container status \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": rpc error: code = NotFound desc = could not find container \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": container with ID starting with 0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51 not found: ID does not exist"
Jul 25 01:14:48.924300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.924261    2562 scope.go:117] "RemoveContainer" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"
Jul 25 01:14:48.924473 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:14:48.924456    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": container with ID starting with e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628 not found: ID does not exist" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"
Jul 25 01:14:48.924513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.924480    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"} err="failed to get container status \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": rpc error: code = NotFound desc = could not find container \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": container with ID starting with e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628 not found: ID does not exist"
Jul 25 01:14:48.924513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.924502    2562 scope.go:117] "RemoveContainer" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"
Jul 25 01:14:48.924690 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:14:48.924673    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": container with ID starting with b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea not found: ID does not exist" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"
Jul 25 01:14:48.924732 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.924695    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"} err="failed to get container status \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": rpc error: code = NotFound desc = could not find container \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": container with ID starting with b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea not found: ID does not exist"
Jul 25 01:14:48.924732 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.924707    2562 scope.go:117] "RemoveContainer" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"
Jul 25 01:14:48.924933 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:14:48.924917    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": container with ID starting with c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231 not found: ID does not exist" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"
Jul 25 01:14:48.925019 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.924941    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"} err="failed to get container status \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": rpc error: code = NotFound desc = could not find container \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": container with ID starting with c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231 not found: ID does not exist"
Jul 25 01:14:48.925019 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.924960    2562 scope.go:117] "RemoveContainer" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"
Jul 25 01:14:48.925194 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:14:48.925177    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": container with ID starting with c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498 not found: ID does not exist" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"
Jul 25 01:14:48.925236 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.925198    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"} err="failed to get container status \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": rpc error: code = NotFound desc = could not find container \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": container with ID starting with c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498 not found: ID does not exist"
Jul 25 01:14:48.925236 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.925213    2562 scope.go:117] "RemoveContainer" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"
Jul 25 01:14:48.925422 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:14:48.925407    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": container with ID starting with 7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2 not found: ID does not exist" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"
Jul 25 01:14:48.925475 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.925424    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"} err="failed to get container status \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": rpc error: code = NotFound desc = could not find container \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": container with ID starting with 7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2 not found: ID does not exist"
Jul 25 01:14:48.925475 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.925436    2562 scope.go:117] "RemoveContainer" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"
Jul 25 01:14:48.925646 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.925620    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"} err="failed to get container status \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": rpc error: code = NotFound desc = could not find container \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": container with ID starting with 48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6 not found: ID does not exist"
Jul 25 01:14:48.925694 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.925647    2562 scope.go:117] "RemoveContainer" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"
Jul 25 01:14:48.925857 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.925840    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"} err="failed to get container status \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": rpc error: code = NotFound desc = could not find container \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": container with ID starting with 0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51 not found: ID does not exist"
Jul 25 01:14:48.925900 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.925858    2562 scope.go:117] "RemoveContainer" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"
Jul 25 01:14:48.926126 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.926106    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"} err="failed to get container status \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": rpc error: code = NotFound desc = could not find container \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": container with ID starting with e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628 not found: ID does not exist"
Jul 25 01:14:48.926126 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.926125    2562 scope.go:117] "RemoveContainer" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"
Jul 25 01:14:48.926329 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.926313    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"} err="failed to get container status \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": rpc error: code = NotFound desc = could not find container \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": container with ID starting with b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea not found: ID does not exist"
Jul 25 01:14:48.926395 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.926330    2562 scope.go:117] "RemoveContainer" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"
Jul 25 01:14:48.926554 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.926523    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"} err="failed to get container status \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": rpc error: code = NotFound desc = could not find container \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": container with ID starting with c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231 not found: ID does not exist"
Jul 25 01:14:48.926554 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.926544    2562 scope.go:117] "RemoveContainer" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"
Jul 25 01:14:48.926865 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.926829    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"} err="failed to get container status \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": rpc error: code = NotFound desc = could not find container \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": container with ID starting with c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498 not found: ID does not exist"
Jul 25 01:14:48.926865 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.926862    2562 scope.go:117] "RemoveContainer" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"
Jul 25 01:14:48.927210 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.927183    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"} err="failed to get container status \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": rpc error: code = NotFound desc = could not find container \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": container with ID starting with 7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2 not found: ID does not exist"
Jul 25 01:14:48.927210 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.927209    2562 scope.go:117] "RemoveContainer" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"
Jul 25 01:14:48.927515 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.927483    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"} err="failed to get container status \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": rpc error: code = NotFound desc = could not find container \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": container with ID starting with 48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6 not found: ID does not exist"
Jul 25 01:14:48.927607 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.927518    2562 scope.go:117] "RemoveContainer" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"
Jul 25 01:14:48.927787 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.927770    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"} err="failed to get container status \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": rpc error: code = NotFound desc = could not find container \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": container with ID starting with 0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51 not found: ID does not exist"
Jul 25 01:14:48.927787 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.927786    2562 scope.go:117] "RemoveContainer" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"
Jul 25 01:14:48.928017 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.927997    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"} err="failed to get container status \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": rpc error: code = NotFound desc = could not find container \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": container with ID starting with e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628 not found: ID does not exist"
Jul 25 01:14:48.928017 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928018    2562 scope.go:117] "RemoveContainer" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"
Jul 25 01:14:48.928262 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928241    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"} err="failed to get container status \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": rpc error: code = NotFound desc = could not find container \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": container with ID starting with b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea not found: ID does not exist"
Jul 25 01:14:48.928314 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928264    2562 scope.go:117] "RemoveContainer" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"
Jul 25 01:14:48.928471 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928454    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:14:48.928549 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928457    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"} err="failed to get container status \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": rpc error: code = NotFound desc = could not find container \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": container with ID starting with c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231 not found: ID does not exist"
Jul 25 01:14:48.928587 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928553    2562 scope.go:117] "RemoveContainer" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"
Jul 25 01:14:48.928744 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928730    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="init-config-reloader"
Jul 25 01:14:48.928796 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928744    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="init-config-reloader"
Jul 25 01:14:48.928796 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928754    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="prometheus"
Jul 25 01:14:48.928796 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928763    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="prometheus"
Jul 25 01:14:48.928796 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928765    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"} err="failed to get container status \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": rpc error: code = NotFound desc = could not find container \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": container with ID starting with c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498 not found: ID does not exist"
Jul 25 01:14:48.928796 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928784    2562 scope.go:117] "RemoveContainer" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928773    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="thanos-sidecar"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928833    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="thanos-sidecar"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928843    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy-web"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928849    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy-web"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928859    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="config-reloader"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928865    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="config-reloader"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928878    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928885    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928892    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy-thanos"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928897    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy-thanos"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928956    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="config-reloader"
Jul 25 01:14:48.928964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928965    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="thanos-sidecar"
Jul 25 01:14:48.929387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.928992    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy-thanos"
Jul 25 01:14:48.929387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929001    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy"
Jul 25 01:14:48.929387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929011    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="kube-rbac-proxy-web"
Jul 25 01:14:48.929387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929018    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" containerName="prometheus"
Jul 25 01:14:48.929387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929024    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"} err="failed to get container status \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": rpc error: code = NotFound desc = could not find container \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": container with ID starting with 7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2 not found: ID does not exist"
Jul 25 01:14:48.929387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929038    2562 scope.go:117] "RemoveContainer" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"
Jul 25 01:14:48.929387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929225    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"} err="failed to get container status \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": rpc error: code = NotFound desc = could not find container \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": container with ID starting with 48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6 not found: ID does not exist"
Jul 25 01:14:48.929387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929241    2562 scope.go:117] "RemoveContainer" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"
Jul 25 01:14:48.929725 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929433    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"} err="failed to get container status \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": rpc error: code = NotFound desc = could not find container \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": container with ID starting with 0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51 not found: ID does not exist"
Jul 25 01:14:48.929725 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929451    2562 scope.go:117] "RemoveContainer" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"
Jul 25 01:14:48.929725 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929706    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"} err="failed to get container status \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": rpc error: code = NotFound desc = could not find container \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": container with ID starting with e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628 not found: ID does not exist"
Jul 25 01:14:48.929725 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929719    2562 scope.go:117] "RemoveContainer" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"
Jul 25 01:14:48.929961 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929944    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"} err="failed to get container status \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": rpc error: code = NotFound desc = could not find container \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": container with ID starting with b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea not found: ID does not exist"
Jul 25 01:14:48.930025 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.929961    2562 scope.go:117] "RemoveContainer" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"
Jul 25 01:14:48.930179 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930161    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"} err="failed to get container status \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": rpc error: code = NotFound desc = could not find container \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": container with ID starting with c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231 not found: ID does not exist"
Jul 25 01:14:48.930222 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930179    2562 scope.go:117] "RemoveContainer" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"
Jul 25 01:14:48.930392 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930378    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"} err="failed to get container status \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": rpc error: code = NotFound desc = could not find container \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": container with ID starting with c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498 not found: ID does not exist"
Jul 25 01:14:48.930437 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930392    2562 scope.go:117] "RemoveContainer" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"
Jul 25 01:14:48.930613 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930598    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"} err="failed to get container status \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": rpc error: code = NotFound desc = could not find container \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": container with ID starting with 7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2 not found: ID does not exist"
Jul 25 01:14:48.930673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930614    2562 scope.go:117] "RemoveContainer" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"
Jul 25 01:14:48.930808 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930793    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"} err="failed to get container status \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": rpc error: code = NotFound desc = could not find container \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": container with ID starting with 48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6 not found: ID does not exist"
Jul 25 01:14:48.930845 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930808    2562 scope.go:117] "RemoveContainer" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"
Jul 25 01:14:48.931019 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.930999    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"} err="failed to get container status \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": rpc error: code = NotFound desc = could not find container \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": container with ID starting with 0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51 not found: ID does not exist"
Jul 25 01:14:48.931019 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931017    2562 scope.go:117] "RemoveContainer" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"
Jul 25 01:14:48.931249 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931230    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"} err="failed to get container status \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": rpc error: code = NotFound desc = could not find container \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": container with ID starting with e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628 not found: ID does not exist"
Jul 25 01:14:48.931325 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931251    2562 scope.go:117] "RemoveContainer" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"
Jul 25 01:14:48.931502 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931486    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"} err="failed to get container status \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": rpc error: code = NotFound desc = could not find container \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": container with ID starting with b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea not found: ID does not exist"
Jul 25 01:14:48.931548 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931503    2562 scope.go:117] "RemoveContainer" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"
Jul 25 01:14:48.931691 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931676    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"} err="failed to get container status \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": rpc error: code = NotFound desc = could not find container \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": container with ID starting with c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231 not found: ID does not exist"
Jul 25 01:14:48.931735 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931694    2562 scope.go:117] "RemoveContainer" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"
Jul 25 01:14:48.931890 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931875    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"} err="failed to get container status \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": rpc error: code = NotFound desc = could not find container \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": container with ID starting with c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498 not found: ID does not exist"
Jul 25 01:14:48.931934 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.931890    2562 scope.go:117] "RemoveContainer" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"
Jul 25 01:14:48.932104 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932086    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"} err="failed to get container status \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": rpc error: code = NotFound desc = could not find container \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": container with ID starting with 7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2 not found: ID does not exist"
Jul 25 01:14:48.932161 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932106    2562 scope.go:117] "RemoveContainer" containerID="48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"
Jul 25 01:14:48.932314 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932295    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6"} err="failed to get container status \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": rpc error: code = NotFound desc = could not find container \"48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6\": container with ID starting with 48485b305bb169f0ae1f3ff5439c449b990461c194596df15c793ac93aca48f6 not found: ID does not exist"
Jul 25 01:14:48.932383 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932315    2562 scope.go:117] "RemoveContainer" containerID="0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"
Jul 25 01:14:48.932546 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932530    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51"} err="failed to get container status \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": rpc error: code = NotFound desc = could not find container \"0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51\": container with ID starting with 0b1c1a346433b69433abdaee25739218e0e359fa89b6280a7db0ba779ebd1b51 not found: ID does not exist"
Jul 25 01:14:48.932608 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932547    2562 scope.go:117] "RemoveContainer" containerID="e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"
Jul 25 01:14:48.932721 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932699    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628"} err="failed to get container status \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": rpc error: code = NotFound desc = could not find container \"e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628\": container with ID starting with e39658491c7f789a54b1f2a766295905bf1b1f0f9ee2ae58a03611a41ab55628 not found: ID does not exist"
Jul 25 01:14:48.932721 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932715    2562 scope.go:117] "RemoveContainer" containerID="b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"
Jul 25 01:14:48.932905 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932889    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea"} err="failed to get container status \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": rpc error: code = NotFound desc = could not find container \"b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea\": container with ID starting with b07f2172463e79cd4f4bf24bb64bd65874747272326febc1abc6edb74a3848ea not found: ID does not exist"
Jul 25 01:14:48.932969 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.932908    2562 scope.go:117] "RemoveContainer" containerID="c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"
Jul 25 01:14:48.933109 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.933095    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231"} err="failed to get container status \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": rpc error: code = NotFound desc = could not find container \"c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231\": container with ID starting with c8a1dcae89967f8c2cff1e6ed486f2568f0eaa007431c98da4ab763ba160c231 not found: ID does not exist"
Jul 25 01:14:48.933154 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.933109    2562 scope.go:117] "RemoveContainer" containerID="c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"
Jul 25 01:14:48.933303 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.933287    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498"} err="failed to get container status \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": rpc error: code = NotFound desc = could not find container \"c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498\": container with ID starting with c0d013d3399ec39f20d19862d6ca959ad5096df8747b0cd5044da63fa556d498 not found: ID does not exist"
Jul 25 01:14:48.933362 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.933304    2562 scope.go:117] "RemoveContainer" containerID="7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"
Jul 25 01:14:48.933503 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.933487    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2"} err="failed to get container status \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": rpc error: code = NotFound desc = could not find container \"7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2\": container with ID starting with 7c6c6878b0b2b4015769fe1249ce5407327a65fef57a7b6b01f490c6b401b5b2 not found: ID does not exist"
Jul 25 01:14:48.934116 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.934104    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:48.936910 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.936893    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Jul 25 01:14:48.937071 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.936955    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Jul 25 01:14:48.937258 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.937242    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ppv7x\""
Jul 25 01:14:48.937323 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.937305    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Jul 25 01:14:48.937472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.937458    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Jul 25 01:14:48.938339 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938324    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Jul 25 01:14:48.938412 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938395    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Jul 25 01:14:48.938474 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938400    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Jul 25 01:14:48.938529 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938513    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Jul 25 01:14:48.938734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938712    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Jul 25 01:14:48.938734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938723    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Jul 25 01:14:48.938881 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938767    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Jul 25 01:14:48.938881 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938726    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Jul 25 01:14:48.938881 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.938713    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Jul 25 01:14:48.939159 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.939143    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-4vcvtk199ss30\""
Jul 25 01:14:48.940935 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.940915    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Jul 25 01:14:48.942891 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.942876    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Jul 25 01:14:48.947935 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:48.947917    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:14:49.012094 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012039    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012094 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012068    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012099    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012174    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012210    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4dc8a3d3-576d-4418-8956-0a1496f83485-config-out\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012233    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012300    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012327    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4bk7\" (UniqueName: \"kubernetes.io/projected/4dc8a3d3-576d-4418-8956-0a1496f83485-kube-api-access-j4bk7\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012352    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-config\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012366    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4dc8a3d3-576d-4418-8956-0a1496f83485-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012383    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012403    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012423    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-web-config\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012442    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012466    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012522    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012555    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.012635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.012572    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.113767 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113711    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.113767 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113770    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113796    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113834    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113860    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113897    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113919    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113946    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.113967    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114017    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4dc8a3d3-576d-4418-8956-0a1496f83485-config-out\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114045    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114095    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j4bk7\" (UniqueName: \"kubernetes.io/projected/4dc8a3d3-576d-4418-8956-0a1496f83485-kube-api-access-j4bk7\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114186    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-config\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114209    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4dc8a3d3-576d-4418-8956-0a1496f83485-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114235    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114262    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114285    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-web-config\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114308    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114339    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114680    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114699    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.114912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114718    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.115083 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.114940    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.115622 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.115598    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.117237 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.117038    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.117237 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.117140    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.117237 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.117159    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.117237 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.117167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-config\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.117744 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.117716    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.117837 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.117729    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.117899 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.117859    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.117952 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.117926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.118286 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.118263    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.118450 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.118427    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/4dc8a3d3-576d-4418-8956-0a1496f83485-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.119199 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.119175    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-web-config\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.119315 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.119298    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/4dc8a3d3-576d-4418-8956-0a1496f83485-config-out\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.119788 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.119767    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/4dc8a3d3-576d-4418-8956-0a1496f83485-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.120502 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.120485    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/4dc8a3d3-576d-4418-8956-0a1496f83485-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.125191 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.125176    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4bk7\" (UniqueName: \"kubernetes.io/projected/4dc8a3d3-576d-4418-8956-0a1496f83485-kube-api-access-j4bk7\") pod \"prometheus-k8s-0\" (UID: \"4dc8a3d3-576d-4418-8956-0a1496f83485\") " pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.243274 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.243251    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:14:49.375122 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.375101    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Jul 25 01:14:49.377393 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:14:49.377361    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4dc8a3d3_576d_4418_8956_0a1496f83485.slice/crio-9146c981485f88972fb2f5b2862a4636068afa3c050cded9c4e8bdb896b784c0 WatchSource:0}: Error finding container 9146c981485f88972fb2f5b2862a4636068afa3c050cded9c4e8bdb896b784c0: Status 404 returned error can't find the container with id 9146c981485f88972fb2f5b2862a4636068afa3c050cded9c4e8bdb896b784c0
Jul 25 01:14:49.881027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.880992    2562 generic.go:356] "Generic (PLEG): container finished" podID="4dc8a3d3-576d-4418-8956-0a1496f83485" containerID="9452d46f137c87baddfb6ee0ce7a1dd69005d6621a57054c6dfd757f6fa9cbf0" exitCode=0
Jul 25 01:14:49.881472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.881074    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4dc8a3d3-576d-4418-8956-0a1496f83485","Type":"ContainerDied","Data":"9452d46f137c87baddfb6ee0ce7a1dd69005d6621a57054c6dfd757f6fa9cbf0"}
Jul 25 01:14:49.881472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:49.881201    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4dc8a3d3-576d-4418-8956-0a1496f83485","Type":"ContainerStarted","Data":"9146c981485f88972fb2f5b2862a4636068afa3c050cded9c4e8bdb896b784c0"}
Jul 25 01:14:50.078211 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:50.078181    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd7e9f3a-425e-4e05-9276-d58bc415bf18" path="/var/lib/kubelet/pods/dd7e9f3a-425e-4e05-9276-d58bc415bf18/volumes"
Jul 25 01:14:50.890179 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:50.890146    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4dc8a3d3-576d-4418-8956-0a1496f83485","Type":"ContainerStarted","Data":"77679eac25eee0a1d5496c7fd327525c3cab6e7e41833635894cb3b2cb25f1dc"}
Jul 25 01:14:50.890538 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:50.890206    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4dc8a3d3-576d-4418-8956-0a1496f83485","Type":"ContainerStarted","Data":"daf4423b1a75edf51b40dcb175d6012b2381230912b5080ca3c611238959c968"}
Jul 25 01:14:50.890538 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:50.890234    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4dc8a3d3-576d-4418-8956-0a1496f83485","Type":"ContainerStarted","Data":"acccc9a8a11e011c51d2dc012a3a706241535e546192ea12bba9340a4963cd50"}
Jul 25 01:14:50.890538 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:50.890257    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4dc8a3d3-576d-4418-8956-0a1496f83485","Type":"ContainerStarted","Data":"1cb54a5fcc643c75f1868d257acfe148ae8f1c732c65b05dcb7f3a22aa168290"}
Jul 25 01:14:50.890538 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:50.890280    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4dc8a3d3-576d-4418-8956-0a1496f83485","Type":"ContainerStarted","Data":"33a5d577d6d291f75eb6aca73c3262569aa86afc868e9b2dba78f727b89322b0"}
Jul 25 01:14:50.890538 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:50.890305    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"4dc8a3d3-576d-4418-8956-0a1496f83485","Type":"ContainerStarted","Data":"1e193405ff34c68b76c866fdd2c781434fe7d51609a92fe4258b80075d2cedb0"}
Jul 25 01:14:50.921701 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:50.921652    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.902137696 podStartE2EDuration="2.921630874s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.493178ms" isStatefulPod=false podCreationTimestamp="2026-07-25 01:14:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:14:50.919325577 +0000 UTC m=+169.390281920" watchObservedRunningTime="2026-07-25 01:14:50.921630874 +0000 UTC m=+169.392587268"
Jul 25 01:14:51.705667 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:51.705632    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:51.819208 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:51.819181    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:52.355272 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:52.355240    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:52.404276 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:52.404244    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:14:54.244037 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:14:54.243947    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:15:02.353858 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:02.353780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:02.454392 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:02.454355    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:16.344667 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:16.344630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:16.446064 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:16.446036    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:49.243715 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:49.243670    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:15:49.258592 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:49.258566    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:15:50.103523 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:50.103500    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Jul 25 01:15:58.257491 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:58.257452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:15:58.312475 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:15:58.312438    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:07.446780 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:07.446738    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:10.238491 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:10.238449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:10.339480 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:10.339442    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:12.358855 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:12.358816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:12.413077 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:12.413049    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:18.804990 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:18.804952    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:23.587673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:23.587582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:23.642059 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:23.642024    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:23.701763 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:23.701726    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:27.159071 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:27.159032    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:16:27.257082 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:16:27.257043    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:01.885007 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:01.884969    2562 kubelet.go:1648] "Image garbage collection succeeded"
Jul 25 01:17:10.775268 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:10.775234    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:12.013239 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:12.013200    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:37.444718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:37.444688    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:38.256329 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:38.256301    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:41.070586 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:41.070550    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:41.170875 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:41.170845    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:43.013376 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:43.013334    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:43.149569 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:43.149525    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:43.298783 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:43.298709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:43.415919 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:43.415889    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:45.678303 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:45.678262    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:45.858846 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:45.858807    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:47.847945 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:47.847906    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:47.901871 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:47.901839    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:51.845053 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:51.845023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:51.946932 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:51.946904    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:53.268387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:53.268352    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:57.296238 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:57.296203    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:17:57.372444 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:17:57.372412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:02.470750 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:02.470715    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:02.570810 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:02.570771    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:04.375614 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:04.375582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:16.265678 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:16.265647    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:16.358373 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:16.358345    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:16.706637 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:16.706593    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:16.803148 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:16.803107    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:16.895461 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:16.895437    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:17.258581 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.258540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:17.364342 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.364311    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:17.379912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.379885    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"]
Jul 25 01:18:17.382954 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.382938    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.386126 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.386103    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Jul 25 01:18:17.386427 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.386407    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Jul 25 01:18:17.386515 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.386439    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Jul 25 01:18:17.386515 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.386441    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Jul 25 01:18:17.386846 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.386829    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ndvlx\""
Jul 25 01:18:17.398877 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.398849    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"]
Jul 25 01:18:17.440284 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.440256    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h42j5\" (UniqueName: \"kubernetes.io/projected/bf113b42-7ff9-4159-89d4-429204ee9bf4-kube-api-access-h42j5\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.440442 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.440319    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bf113b42-7ff9-4159-89d4-429204ee9bf4-manifests\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.440442 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.440349    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf113b42-7ff9-4159-89d4-429204ee9bf4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.440442 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.440419    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf113b42-7ff9-4159-89d4-429204ee9bf4-tmp\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.440576 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.440451    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bf113b42-7ff9-4159-89d4-429204ee9bf4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.524737 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.524655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:17.541731 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.541710    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bf113b42-7ff9-4159-89d4-429204ee9bf4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.541798 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.541747    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h42j5\" (UniqueName: \"kubernetes.io/projected/bf113b42-7ff9-4159-89d4-429204ee9bf4-kube-api-access-h42j5\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.541861 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.541850    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bf113b42-7ff9-4159-89d4-429204ee9bf4-manifests\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.541902 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.541869    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf113b42-7ff9-4159-89d4-429204ee9bf4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.541943 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.541907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf113b42-7ff9-4159-89d4-429204ee9bf4-tmp\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.542259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.542240    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bf113b42-7ff9-4159-89d4-429204ee9bf4-manifests\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.544131 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.544103    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bf113b42-7ff9-4159-89d4-429204ee9bf4-tmp\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.544251 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.544234    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bf113b42-7ff9-4159-89d4-429204ee9bf4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.544312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.544297    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bf113b42-7ff9-4159-89d4-429204ee9bf4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.550995 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.550951    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h42j5\" (UniqueName: \"kubernetes.io/projected/bf113b42-7ff9-4159-89d4-429204ee9bf4-kube-api-access-h42j5\") pod \"opendatahub-operator-controller-manager-7587ff6c45-m2xxg\" (UID: \"bf113b42-7ff9-4159-89d4-429204ee9bf4\") " pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.632799 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.632777    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:17.693855 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.693837    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:17.735990 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.735951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:17.826103 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.826083    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"]
Jul 25 01:18:17.828204 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:18:17.828181    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf113b42_7ff9_4159_89d4_429204ee9bf4.slice/crio-c2fa779cc4a447ff5fedaeb318cc548fc178e174b2f334f43891936eeceb0735 WatchSource:0}: Error finding container c2fa779cc4a447ff5fedaeb318cc548fc178e174b2f334f43891936eeceb0735: Status 404 returned error can't find the container with id c2fa779cc4a447ff5fedaeb318cc548fc178e174b2f334f43891936eeceb0735
Jul 25 01:18:17.830177 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.830160    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:18:17.834776 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.834754    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:17.902422 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:17.902386    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:18.008299 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:18.008274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:18.111713 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:18.111655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:18.578015 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:18.577926    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg" event={"ID":"bf113b42-7ff9-4159-89d4-429204ee9bf4","Type":"ContainerStarted","Data":"c2fa779cc4a447ff5fedaeb318cc548fc178e174b2f334f43891936eeceb0735"}
Jul 25 01:18:21.592261 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:21.592225    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg" event={"ID":"bf113b42-7ff9-4159-89d4-429204ee9bf4","Type":"ContainerStarted","Data":"77ee0b87997904c1956950fd479d318ed122ece370d9d0aac18a09f919c40bc6"}
Jul 25 01:18:22.596545 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:22.596510    2562 generic.go:356] "Generic (PLEG): container finished" podID="bf113b42-7ff9-4159-89d4-429204ee9bf4" containerID="77ee0b87997904c1956950fd479d318ed122ece370d9d0aac18a09f919c40bc6" exitCode=0
Jul 25 01:18:22.596916 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:22.596598    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg" event={"ID":"bf113b42-7ff9-4159-89d4-429204ee9bf4","Type":"ContainerDied","Data":"77ee0b87997904c1956950fd479d318ed122ece370d9d0aac18a09f919c40bc6"}
Jul 25 01:18:23.602105 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:23.602067    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg" event={"ID":"bf113b42-7ff9-4159-89d4-429204ee9bf4","Type":"ContainerStarted","Data":"72b615742a8e7dc09bce6a5d6521463e0f51af0a7fd0b38cd5c9699348ea1ba5"}
Jul 25 01:18:23.602481 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:23.602184    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:23.624676 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:23.624627    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg" podStartSLOduration=2.128440771 podStartE2EDuration="6.624612365s" totalImagesPullingTime="3.442413393s" totalInitContainerRuntime="1.053758201s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:17 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:23.62253268 +0000 UTC m=+382.093489017" watchObservedRunningTime="2026-07-25 01:18:23.624612365 +0000 UTC m=+382.095568704"
Jul 25 01:18:34.607747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:34.607721    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7587ff6c45-m2xxg"
Jul 25 01:18:37.477256 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:37.477222    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:37.592720 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:37.592675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:37.978666 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:37.978633    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:38.161450 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.161409    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"]
Jul 25 01:18:38.169520 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.169494    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.170725 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.170701    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pmv72"]
Jul 25 01:18:38.172832 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.172616    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Jul 25 01:18:38.172832 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.172736    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-9hjgq\""
Jul 25 01:18:38.172832 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.172761    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Jul 25 01:18:38.173075 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.172924    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:38.179586 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.179562    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Jul 25 01:18:38.179702 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.179626    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Jul 25 01:18:38.179914 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.179890    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-ltbcv\""
Jul 25 01:18:38.180133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.179961    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Jul 25 01:18:38.182890 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.182864    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"]
Jul 25 01:18:38.189280 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.189258    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pmv72"]
Jul 25 01:18:38.220897 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.220868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:38.290294 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.290199    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fw4zn\" (UniqueName: \"kubernetes.io/projected/ab660b12-c049-4d2d-a814-8247e818ebc2-kube-api-access-fw4zn\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.290294 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.290267    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-cert\") pod \"odh-model-controller-7998c67dc4-pmv72\" (UID: \"dca5076c-ffa3-4fd0-a8cb-227a91400d8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:38.290512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.290295    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rr64l\" (UniqueName: \"kubernetes.io/projected/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-kube-api-access-rr64l\") pod \"odh-model-controller-7998c67dc4-pmv72\" (UID: \"dca5076c-ffa3-4fd0-a8cb-227a91400d8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:38.290512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.290393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab660b12-c049-4d2d-a814-8247e818ebc2-tls-certs\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.290512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.290443    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab660b12-c049-4d2d-a814-8247e818ebc2-tmp\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.317849 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.317819    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:38.391124 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.391082    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-cert\") pod \"odh-model-controller-7998c67dc4-pmv72\" (UID: \"dca5076c-ffa3-4fd0-a8cb-227a91400d8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:38.391124 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.391121    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rr64l\" (UniqueName: \"kubernetes.io/projected/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-kube-api-access-rr64l\") pod \"odh-model-controller-7998c67dc4-pmv72\" (UID: \"dca5076c-ffa3-4fd0-a8cb-227a91400d8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:38.391329 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.391173    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab660b12-c049-4d2d-a814-8247e818ebc2-tls-certs\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.391329 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.391219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab660b12-c049-4d2d-a814-8247e818ebc2-tmp\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.391329 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:18:38.391250    2562 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Jul 25 01:18:38.391464 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:18:38.391326    2562 secret.go:189] Couldn't get secret openshift-ingress/kube-auth-proxy-tls: secret "kube-auth-proxy-tls" not found
Jul 25 01:18:38.391464 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.391251    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fw4zn\" (UniqueName: \"kubernetes.io/projected/ab660b12-c049-4d2d-a814-8247e818ebc2-kube-api-access-fw4zn\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.391464 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:18:38.391332    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-cert podName:dca5076c-ffa3-4fd0-a8cb-227a91400d8f nodeName:}" failed. No retries permitted until 2026-07-25 01:18:38.891310671 +0000 UTC m=+397.362266990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-cert") pod "odh-model-controller-7998c67dc4-pmv72" (UID: "dca5076c-ffa3-4fd0-a8cb-227a91400d8f") : secret "odh-model-controller-webhook-cert" not found
Jul 25 01:18:38.391464 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:18:38.391412    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab660b12-c049-4d2d-a814-8247e818ebc2-tls-certs podName:ab660b12-c049-4d2d-a814-8247e818ebc2 nodeName:}" failed. No retries permitted until 2026-07-25 01:18:38.89139565 +0000 UTC m=+397.362351967 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/ab660b12-c049-4d2d-a814-8247e818ebc2-tls-certs") pod "kube-auth-proxy-789f49c94d-rq8fr" (UID: "ab660b12-c049-4d2d-a814-8247e818ebc2") : secret "kube-auth-proxy-tls" not found
Jul 25 01:18:38.393375 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.393351    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ab660b12-c049-4d2d-a814-8247e818ebc2-tmp\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.400609 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.400580    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rr64l\" (UniqueName: \"kubernetes.io/projected/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-kube-api-access-rr64l\") pod \"odh-model-controller-7998c67dc4-pmv72\" (UID: \"dca5076c-ffa3-4fd0-a8cb-227a91400d8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:38.401674 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.401654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fw4zn\" (UniqueName: \"kubernetes.io/projected/ab660b12-c049-4d2d-a814-8247e818ebc2-kube-api-access-fw4zn\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.421353 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.421324    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:38.519737 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.519703    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:38.895124 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.895089    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab660b12-c049-4d2d-a814-8247e818ebc2-tls-certs\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.895292 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.895154    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-cert\") pod \"odh-model-controller-7998c67dc4-pmv72\" (UID: \"dca5076c-ffa3-4fd0-a8cb-227a91400d8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:38.897595 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.897565    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ab660b12-c049-4d2d-a814-8247e818ebc2-tls-certs\") pod \"kube-auth-proxy-789f49c94d-rq8fr\" (UID: \"ab660b12-c049-4d2d-a814-8247e818ebc2\") " pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:38.897735 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:38.897654    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dca5076c-ffa3-4fd0-a8cb-227a91400d8f-cert\") pod \"odh-model-controller-7998c67dc4-pmv72\" (UID: \"dca5076c-ffa3-4fd0-a8cb-227a91400d8f\") " pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:39.090179 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.090140    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"
Jul 25 01:18:39.096541 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.096517    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:39.126713 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.126676    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nzz8d"]
Jul 25 01:18:39.128842 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.128820    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:39.131925 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.131854    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-thvlt\""
Jul 25 01:18:39.132087 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.132072    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\""
Jul 25 01:18:39.134750 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.134716    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:39.144409 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.144317    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nzz8d"]
Jul 25 01:18:39.197497 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.197458    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"]
Jul 25 01:18:39.197660 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.197637    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkh92\" (UniqueName: \"kubernetes.io/projected/ba9d170c-06a0-4247-976f-611bcbe95ff3-kube-api-access-rkh92\") pod \"model-serving-api-54d89bd5fc-nzz8d\" (UID: \"ba9d170c-06a0-4247-976f-611bcbe95ff3\") " pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:39.197717 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.197680    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba9d170c-06a0-4247-976f-611bcbe95ff3-tls-certs\") pod \"model-serving-api-54d89bd5fc-nzz8d\" (UID: \"ba9d170c-06a0-4247-976f-611bcbe95ff3\") " pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:39.199700 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.199497    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.202662 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.202639    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Jul 25 01:18:39.202902 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.202883    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Jul 25 01:18:39.203270 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.203095    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-jb7sl\""
Jul 25 01:18:39.203270 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.203100    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Jul 25 01:18:39.203270 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.203166    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Jul 25 01:18:39.212437 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.212386    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"]
Jul 25 01:18:39.222906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.222800    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:39.251407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.251383    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr"]
Jul 25 01:18:39.254395 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:18:39.254372    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab660b12_c049_4d2d_a814_8247e818ebc2.slice/crio-f4a6e0af0cd11cb9ae13f0d07a4e2aef40c11b4568a75bd7e5ae9008dac86e18 WatchSource:0}: Error finding container f4a6e0af0cd11cb9ae13f0d07a4e2aef40c11b4568a75bd7e5ae9008dac86e18: Status 404 returned error can't find the container with id f4a6e0af0cd11cb9ae13f0d07a4e2aef40c11b4568a75bd7e5ae9008dac86e18
Jul 25 01:18:39.297075 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.297051    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:39.297994 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.297963    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.298096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.298012    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkh92\" (UniqueName: \"kubernetes.io/projected/ba9d170c-06a0-4247-976f-611bcbe95ff3-kube-api-access-rkh92\") pod \"model-serving-api-54d89bd5fc-nzz8d\" (UID: \"ba9d170c-06a0-4247-976f-611bcbe95ff3\") " pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:39.298096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.298036    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.298096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.298054    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba9d170c-06a0-4247-976f-611bcbe95ff3-tls-certs\") pod \"model-serving-api-54d89bd5fc-nzz8d\" (UID: \"ba9d170c-06a0-4247-976f-611bcbe95ff3\") " pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:39.298096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.298069    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e527177c-0b8b-4395-a405-28c1c45bb364-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.298096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.298086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.298347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.298124    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.298347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.298153    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.298347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.298187    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2pblb\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-kube-api-access-2pblb\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.300495 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.300474    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ba9d170c-06a0-4247-976f-611bcbe95ff3-tls-certs\") pod \"model-serving-api-54d89bd5fc-nzz8d\" (UID: \"ba9d170c-06a0-4247-976f-611bcbe95ff3\") " pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:39.310719 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.310699    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkh92\" (UniqueName: \"kubernetes.io/projected/ba9d170c-06a0-4247-976f-611bcbe95ff3-kube-api-access-rkh92\") pod \"model-serving-api-54d89bd5fc-nzz8d\" (UID: \"ba9d170c-06a0-4247-976f-611bcbe95ff3\") " pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:39.399420 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.399346    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.399420 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.399383    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.399420 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.399402    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e527177c-0b8b-4395-a405-28c1c45bb364-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.399631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.399425    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.399631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.399473    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.399631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.399514    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.399631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.399549    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2pblb\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-kube-api-access-2pblb\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.400102 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.400077    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.401792 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.401770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.402027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.401997    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e527177c-0b8b-4395-a405-28c1c45bb364-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.402306 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.402277    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.402376 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.402323    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.407273 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.407256    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.407464 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.407448    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2pblb\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-kube-api-access-2pblb\") pod \"istiod-openshift-gateway-7b7995d5cd-rqd4p\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.442271 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.442249    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:39.477261 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.477237    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-pmv72"]
Jul 25 01:18:39.480020 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:18:39.479993    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca5076c_ffa3_4fd0_a8cb_227a91400d8f.slice/crio-ab6efa3d0cf84c83a37ae0c14736edcb42696c2151c82dc49522eb1d51469be0 WatchSource:0}: Error finding container ab6efa3d0cf84c83a37ae0c14736edcb42696c2151c82dc49522eb1d51469be0: Status 404 returned error can't find the container with id ab6efa3d0cf84c83a37ae0c14736edcb42696c2151c82dc49522eb1d51469be0
Jul 25 01:18:39.510510 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.510488    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:39.582360 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.582286    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-nzz8d"]
Jul 25 01:18:39.652555 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.652465    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"]
Jul 25 01:18:39.656431 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:18:39.656403    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode527177c_0b8b_4395_a405_28c1c45bb364.slice/crio-8d776e7e77a8d704c87680ac5166acb3d47816a0b7bc126a94b7fc760a511afa WatchSource:0}: Error finding container 8d776e7e77a8d704c87680ac5166acb3d47816a0b7bc126a94b7fc760a511afa: Status 404 returned error can't find the container with id 8d776e7e77a8d704c87680ac5166acb3d47816a0b7bc126a94b7fc760a511afa
Jul 25 01:18:39.659735 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.659709    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pmv72" event={"ID":"dca5076c-ffa3-4fd0-a8cb-227a91400d8f","Type":"ContainerStarted","Data":"ab6efa3d0cf84c83a37ae0c14736edcb42696c2151c82dc49522eb1d51469be0"}
Jul 25 01:18:39.660828 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.660800    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr" event={"ID":"ab660b12-c049-4d2d-a814-8247e818ebc2","Type":"ContainerStarted","Data":"f4a6e0af0cd11cb9ae13f0d07a4e2aef40c11b4568a75bd7e5ae9008dac86e18"}
Jul 25 01:18:39.662114 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:39.662094    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d" event={"ID":"ba9d170c-06a0-4247-976f-611bcbe95ff3","Type":"ContainerStarted","Data":"9c90a9f78b75e089af05a18fa9108ad9888e52368ed48633cdbfd9d5e2443d7d"}
Jul 25 01:18:40.183999 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:40.183842    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:40.671467 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:40.671424    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p" event={"ID":"e527177c-0b8b-4395-a405-28c1c45bb364","Type":"ContainerStarted","Data":"8d776e7e77a8d704c87680ac5166acb3d47816a0b7bc126a94b7fc760a511afa"}
Jul 25 01:18:43.688298 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:43.688259    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d" event={"ID":"ba9d170c-06a0-4247-976f-611bcbe95ff3","Type":"ContainerStarted","Data":"451dc40f8b727d0d3ee548ac826449a7b0c40e79a09e640dc3782f83bbd7fd01"}
Jul 25 01:18:43.688890 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:43.688834    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:43.691075 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:43.691047    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-pmv72" event={"ID":"dca5076c-ffa3-4fd0-a8cb-227a91400d8f","Type":"ContainerStarted","Data":"43576446256c124460c7f3712f11afe08dd1db7907ae43059cd23089a392f009"}
Jul 25 01:18:43.691597 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:43.691552    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:43.693926 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:43.693879    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr" event={"ID":"ab660b12-c049-4d2d-a814-8247e818ebc2","Type":"ContainerStarted","Data":"dbbedb1839be7eacb3abbb7e009efa5286bff700ebea19536239783f5f5bc195"}
Jul 25 01:18:43.710012 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:43.709638    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d" podStartSLOduration=0.937414796 podStartE2EDuration="4.709620295s" totalImagesPullingTime="3.772205499s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:43.707060246 +0000 UTC m=+402.178016585" watchObservedRunningTime="2026-07-25 01:18:43.709620295 +0000 UTC m=+402.180576636"
Jul 25 01:18:43.726327 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:43.726117    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-789f49c94d-rq8fr" podStartSLOduration=1.619276567 podStartE2EDuration="5.726094336s" totalImagesPullingTime="4.106817769s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:43.72475423 +0000 UTC m=+402.195710569" watchObservedRunningTime="2026-07-25 01:18:43.726094336 +0000 UTC m=+402.197050677"
Jul 25 01:18:43.750947 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:43.750748    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-pmv72" podStartSLOduration=1.8262055579999998 podStartE2EDuration="5.750726103s" totalImagesPullingTime="3.924520545s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:43.749108533 +0000 UTC m=+402.220064889" watchObservedRunningTime="2026-07-25 01:18:43.750726103 +0000 UTC m=+402.221682444"
Jul 25 01:18:45.397534 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:45.397495    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:18:45.397794 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:45.397567    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:18:45.703539 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:45.703437    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p" event={"ID":"e527177c-0b8b-4395-a405-28c1c45bb364","Type":"ContainerStarted","Data":"80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23"}
Jul 25 01:18:45.703688 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:45.703617    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:45.725749 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:45.725701    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p" podStartSLOduration=0.98695239 podStartE2EDuration="6.725684313s" totalImagesPullingTime="5.738731923s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:45.724060731 +0000 UTC m=+404.195017071" watchObservedRunningTime="2026-07-25 01:18:45.725684313 +0000 UTC m=+404.196640655"
Jul 25 01:18:46.277632 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:46.277593    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:46.367237 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:46.367202    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:46.708476 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:46.708443    2562 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-rqd4p container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 25 01:18:46.708862 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:46.708501    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p" podUID="e527177c-0b8b-4395-a405-28c1c45bb364" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 25 01:18:46.977004 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:46.976915    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"]
Jul 25 01:18:46.980187 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:46.980168    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:46.982900 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:46.982881    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-z22hw\""
Jul 25 01:18:46.993465 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:46.993443    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"]
Jul 25 01:18:47.011090 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.011065    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:47.073133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073108    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.073297 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073147    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwlxz\" (UniqueName: \"kubernetes.io/projected/1fd0785d-3ce7-469d-9671-bda9d2fd9603-kube-api-access-mwlxz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.073297 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073223    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.073297 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073260    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.073482 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073307    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.073482 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073332    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.073482 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073448    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.073633 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073496    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.073633 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.073597    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.126824 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.126794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:47.174536 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174504    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.174734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174557    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwlxz\" (UniqueName: \"kubernetes.io/projected/1fd0785d-3ce7-469d-9671-bda9d2fd9603-kube-api-access-mwlxz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.174734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174597    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.174734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.174734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174658    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.174734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174680    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.175248 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174740    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.175248 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174786    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.175248 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.174841    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.175248 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.175109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.175248 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.175236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.175527 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.175268    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.175527 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.175288    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.175527 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.175456    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.177164 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.177137    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.177347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.177326    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.186747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.186726    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1fd0785d-3ce7-469d-9671-bda9d2fd9603-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.188122 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.188106    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwlxz\" (UniqueName: \"kubernetes.io/projected/1fd0785d-3ce7-469d-9671-bda9d2fd9603-kube-api-access-mwlxz\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg\" (UID: \"1fd0785d-3ce7-469d-9671-bda9d2fd9603\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.239542 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.239485    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:47.290383 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.290362    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:47.442105 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.442073    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"]
Jul 25 01:18:47.445184 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:18:47.445155    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fd0785d_3ce7_469d_9671_bda9d2fd9603.slice/crio-50bdffef93365564768360decf7a6bb276ef5165f318907754cea9c5db72d3e9 WatchSource:0}: Error finding container 50bdffef93365564768360decf7a6bb276ef5165f318907754cea9c5db72d3e9: Status 404 returned error can't find the container with id 50bdffef93365564768360decf7a6bb276ef5165f318907754cea9c5db72d3e9
Jul 25 01:18:47.712266 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:47.712236    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg" event={"ID":"1fd0785d-3ce7-469d-9671-bda9d2fd9603","Type":"ContainerStarted","Data":"50bdffef93365564768360decf7a6bb276ef5165f318907754cea9c5db72d3e9"}
Jul 25 01:18:49.360088 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.360032    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:49.506362 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.506332    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"]
Jul 25 01:18:49.512506 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.512485    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:49.515795 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.515776    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-xvd74\""
Jul 25 01:18:49.516116 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.516092    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Jul 25 01:18:49.524904 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.524859    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:49.525705 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.525685    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"]
Jul 25 01:18:49.530068 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.530045    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:49.533251 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.533230    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-cc276\""
Jul 25 01:18:49.533363 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.533235    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Jul 25 01:18:49.534197 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.534179    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"]
Jul 25 01:18:49.568229 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.568194    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"]
Jul 25 01:18:49.593878 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.593855    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jc4xw\" (UID: \"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:49.594026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.593904    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldd2w\" (UniqueName: \"kubernetes.io/projected/83ad3d9b-ebc6-4ba0-8469-4ceb60152f68-kube-api-access-ldd2w\") pod \"kserve-controller-manager-d6f4b65b8-9ftn5\" (UID: \"83ad3d9b-ebc6-4ba0-8469-4ceb60152f68\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:49.594026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.593931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83ad3d9b-ebc6-4ba0-8469-4ceb60152f68-cert\") pod \"kserve-controller-manager-d6f4b65b8-9ftn5\" (UID: \"83ad3d9b-ebc6-4ba0-8469-4ceb60152f68\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:49.594146 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.594038    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7wmr\" (UniqueName: \"kubernetes.io/projected/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-kube-api-access-l7wmr\") pod \"llmisvc-controller-manager-777c6bd7d5-jc4xw\" (UID: \"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:49.667487 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.667435    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:49.695318 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.695293    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldd2w\" (UniqueName: \"kubernetes.io/projected/83ad3d9b-ebc6-4ba0-8469-4ceb60152f68-kube-api-access-ldd2w\") pod \"kserve-controller-manager-d6f4b65b8-9ftn5\" (UID: \"83ad3d9b-ebc6-4ba0-8469-4ceb60152f68\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:49.695472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.695331    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83ad3d9b-ebc6-4ba0-8469-4ceb60152f68-cert\") pod \"kserve-controller-manager-d6f4b65b8-9ftn5\" (UID: \"83ad3d9b-ebc6-4ba0-8469-4ceb60152f68\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:49.695472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.695397    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7wmr\" (UniqueName: \"kubernetes.io/projected/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-kube-api-access-l7wmr\") pod \"llmisvc-controller-manager-777c6bd7d5-jc4xw\" (UID: \"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:49.695472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.695463    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jc4xw\" (UID: \"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:49.695623 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:18:49.695582    2562 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Jul 25 01:18:49.695673 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:18:49.695647    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-cert podName:7a6ddf44-2336-4d83-8fdc-fedbb6a9a286 nodeName:}" failed. No retries permitted until 2026-07-25 01:18:50.195625033 +0000 UTC m=+408.666581349 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-cert") pod "llmisvc-controller-manager-777c6bd7d5-jc4xw" (UID: "7a6ddf44-2336-4d83-8fdc-fedbb6a9a286") : secret "llmisvc-webhook-server-cert" not found
Jul 25 01:18:49.698178 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.698155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/83ad3d9b-ebc6-4ba0-8469-4ceb60152f68-cert\") pod \"kserve-controller-manager-d6f4b65b8-9ftn5\" (UID: \"83ad3d9b-ebc6-4ba0-8469-4ceb60152f68\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:49.704656 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.704637    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldd2w\" (UniqueName: \"kubernetes.io/projected/83ad3d9b-ebc6-4ba0-8469-4ceb60152f68-kube-api-access-ldd2w\") pod \"kserve-controller-manager-d6f4b65b8-9ftn5\" (UID: \"83ad3d9b-ebc6-4ba0-8469-4ceb60152f68\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:49.704804 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.704776    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7wmr\" (UniqueName: \"kubernetes.io/projected/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-kube-api-access-l7wmr\") pod \"llmisvc-controller-manager-777c6bd7d5-jc4xw\" (UID: \"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:49.708025 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.708005    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:49.824219 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:49.824188    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:50.037258 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.037225    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:18:50.037350 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.037298    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:18:50.037350 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.037328    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:18:50.114024 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.114001    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"]
Jul 25 01:18:50.116552 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:18:50.116522    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83ad3d9b_ebc6_4ba0_8469_4ceb60152f68.slice/crio-109642699c3c7f14d5c430523bef9e706bbd54a46a36b214c84bb28754ef2ba5 WatchSource:0}: Error finding container 109642699c3c7f14d5c430523bef9e706bbd54a46a36b214c84bb28754ef2ba5: Status 404 returned error can't find the container with id 109642699c3c7f14d5c430523bef9e706bbd54a46a36b214c84bb28754ef2ba5
Jul 25 01:18:50.156425 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.156384    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:50.199888 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.199859    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jc4xw\" (UID: \"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:50.202426 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.202402    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7a6ddf44-2336-4d83-8fdc-fedbb6a9a286-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-jc4xw\" (UID: \"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:50.440675 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.440643    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:50.588890 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.588869    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"]
Jul 25 01:18:50.590753 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:18:50.590729    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7a6ddf44_2336_4d83_8fdc_fedbb6a9a286.slice/crio-7599ebcb730c5fe145cc464638d62d8acd2b8220f916542e231c82ce357eb45b WatchSource:0}: Error finding container 7599ebcb730c5fe145cc464638d62d8acd2b8220f916542e231c82ce357eb45b: Status 404 returned error can't find the container with id 7599ebcb730c5fe145cc464638d62d8acd2b8220f916542e231c82ce357eb45b
Jul 25 01:18:50.727403 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.727322    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg" event={"ID":"1fd0785d-3ce7-469d-9671-bda9d2fd9603","Type":"ContainerStarted","Data":"0e29289c1febf56f6eaa7ea1946887e81b133b900fd69c1e5ce59c1858700755"}
Jul 25 01:18:50.728836 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.728795    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw" event={"ID":"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286","Type":"ContainerStarted","Data":"7599ebcb730c5fe145cc464638d62d8acd2b8220f916542e231c82ce357eb45b"}
Jul 25 01:18:50.730193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.730161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5" event={"ID":"83ad3d9b-ebc6-4ba0-8469-4ceb60152f68","Type":"ContainerStarted","Data":"109642699c3c7f14d5c430523bef9e706bbd54a46a36b214c84bb28754ef2ba5"}
Jul 25 01:18:50.752722 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.752664    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg" podStartSLOduration=2.163274834 podStartE2EDuration="4.752645235s" totalImagesPullingTime="2.589370401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:50.750652581 +0000 UTC m=+409.221608922" watchObservedRunningTime="2026-07-25 01:18:50.752645235 +0000 UTC m=+409.223601573"
Jul 25 01:18:50.761158 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.761135    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:50.861515 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.861484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:50.960953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:50.960845    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:18:51.291437 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:51.291405    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:51.296164 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:51.296140    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:51.735595 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:51.735532    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:51.736756 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:51.736732    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg"
Jul 25 01:18:52.740942 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:52.740862    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5" event={"ID":"83ad3d9b-ebc6-4ba0-8469-4ceb60152f68","Type":"ContainerStarted","Data":"a32da1497944f916340bbb40c4e49d55c3d7a7d18089332da083c1a80fe82ea1"}
Jul 25 01:18:52.741349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:52.741058    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:18:52.758026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:52.757963    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5" podStartSLOduration=1.40467635 podStartE2EDuration="3.757949298s" totalImagesPullingTime="2.353272948s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:52.75694823 +0000 UTC m=+411.227904586" watchObservedRunningTime="2026-07-25 01:18:52.757949298 +0000 UTC m=+411.228905636"
Jul 25 01:18:53.746469 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:53.746438    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw" event={"ID":"7a6ddf44-2336-4d83-8fdc-fedbb6a9a286","Type":"ContainerStarted","Data":"7e451d16c24f100fd586fe3064f286b6032ba237f749c9400d7c8d4822846433"}
Jul 25 01:18:53.746911 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:53.746719    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:18:53.764314 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:53.764267    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw" podStartSLOduration=2.072995388 podStartE2EDuration="4.764253439s" totalImagesPullingTime="2.691258051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:18:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:18:53.762719749 +0000 UTC m=+412.233676117" watchObservedRunningTime="2026-07-25 01:18:53.764253439 +0000 UTC m=+412.235209776"
Jul 25 01:18:53.945360 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:53.945323    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"]
Jul 25 01:18:53.945572 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:53.945548    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p" podUID="e527177c-0b8b-4395-a405-28c1c45bb364" containerName="discovery" containerID="cri-o://80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23" gracePeriod=30
Jul 25 01:18:54.190360 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.190337    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:54.338292 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.338206    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-istio-token\") pod \"e527177c-0b8b-4395-a405-28c1c45bb364\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") "
Jul 25 01:18:54.338292 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.338251    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-kubeconfig\") pod \"e527177c-0b8b-4395-a405-28c1c45bb364\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") "
Jul 25 01:18:54.338501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.338297    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-dns-cert\") pod \"e527177c-0b8b-4395-a405-28c1c45bb364\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") "
Jul 25 01:18:54.338501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.338325    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-cacerts\") pod \"e527177c-0b8b-4395-a405-28c1c45bb364\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") "
Jul 25 01:18:54.338501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.338359    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e527177c-0b8b-4395-a405-28c1c45bb364-local-certs\") pod \"e527177c-0b8b-4395-a405-28c1c45bb364\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") "
Jul 25 01:18:54.338501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.338390    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2pblb\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-kube-api-access-2pblb\") pod \"e527177c-0b8b-4395-a405-28c1c45bb364\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") "
Jul 25 01:18:54.338501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.338454    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-ca-configmap\") pod \"e527177c-0b8b-4395-a405-28c1c45bb364\" (UID: \"e527177c-0b8b-4395-a405-28c1c45bb364\") "
Jul 25 01:18:54.339066 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.339025    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "e527177c-0b8b-4395-a405-28c1c45bb364" (UID: "e527177c-0b8b-4395-a405-28c1c45bb364"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:18:54.340946 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.340908    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-istio-token" (OuterVolumeSpecName: "istio-token") pod "e527177c-0b8b-4395-a405-28c1c45bb364" (UID: "e527177c-0b8b-4395-a405-28c1c45bb364"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:18:54.340946 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.340920    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e527177c-0b8b-4395-a405-28c1c45bb364-local-certs" (OuterVolumeSpecName: "local-certs") pod "e527177c-0b8b-4395-a405-28c1c45bb364" (UID: "e527177c-0b8b-4395-a405-28c1c45bb364"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:18:54.341153 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.340920    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "e527177c-0b8b-4395-a405-28c1c45bb364" (UID: "e527177c-0b8b-4395-a405-28c1c45bb364"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:18:54.341533 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.341509    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-kube-api-access-2pblb" (OuterVolumeSpecName: "kube-api-access-2pblb") pod "e527177c-0b8b-4395-a405-28c1c45bb364" (UID: "e527177c-0b8b-4395-a405-28c1c45bb364"). InnerVolumeSpecName "kube-api-access-2pblb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:18:54.341783 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.341760    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-cacerts" (OuterVolumeSpecName: "cacerts") pod "e527177c-0b8b-4395-a405-28c1c45bb364" (UID: "e527177c-0b8b-4395-a405-28c1c45bb364"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:18:54.341950 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.341932    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "e527177c-0b8b-4395-a405-28c1c45bb364" (UID: "e527177c-0b8b-4395-a405-28c1c45bb364"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:18:54.439791 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.439761    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-ca-configmap\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:18:54.439922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.439797    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-istio-token\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:18:54.439922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.439815    2562 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-kubeconfig\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:18:54.439922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.439829    2562 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-istio-csr-dns-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:18:54.439922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.439844    2562 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/e527177c-0b8b-4395-a405-28c1c45bb364-cacerts\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:18:54.439922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.439859    2562 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/e527177c-0b8b-4395-a405-28c1c45bb364-local-certs\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:18:54.439922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.439872    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2pblb\" (UniqueName: \"kubernetes.io/projected/e527177c-0b8b-4395-a405-28c1c45bb364-kube-api-access-2pblb\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:18:54.752259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.752225    2562 generic.go:356] "Generic (PLEG): container finished" podID="e527177c-0b8b-4395-a405-28c1c45bb364" containerID="80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23" exitCode=0
Jul 25 01:18:54.752670 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.752286    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"
Jul 25 01:18:54.752670 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.752316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p" event={"ID":"e527177c-0b8b-4395-a405-28c1c45bb364","Type":"ContainerDied","Data":"80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23"}
Jul 25 01:18:54.752670 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.752362    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p" event={"ID":"e527177c-0b8b-4395-a405-28c1c45bb364","Type":"ContainerDied","Data":"8d776e7e77a8d704c87680ac5166acb3d47816a0b7bc126a94b7fc760a511afa"}
Jul 25 01:18:54.752670 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.752383    2562 scope.go:117] "RemoveContainer" containerID="80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23"
Jul 25 01:18:54.760778 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.760762    2562 scope.go:117] "RemoveContainer" containerID="80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23"
Jul 25 01:18:54.761082 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:18:54.761061    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23\": container with ID starting with 80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23 not found: ID does not exist" containerID="80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23"
Jul 25 01:18:54.761170 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.761092    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23"} err="failed to get container status \"80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23\": rpc error: code = NotFound desc = could not find container \"80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23\": container with ID starting with 80994aa134536d7609a1ac7b7908b9a96e9117eb7d4b302b9f2b37f1556e3e23 not found: ID does not exist"
Jul 25 01:18:54.785136 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.785100    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"]
Jul 25 01:18:54.786941 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:54.786917    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-rqd4p"]
Jul 25 01:18:55.707508 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:55.707476    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-nzz8d"
Jul 25 01:18:55.737397 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:55.737374    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-pmv72"
Jul 25 01:18:56.075315 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:18:56.075247    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e527177c-0b8b-4395-a405-28c1c45bb364" path="/var/lib/kubelet/pods/e527177c-0b8b-4395-a405-28c1c45bb364/volumes"
Jul 25 01:19:23.753258 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:23.753180    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-9ftn5"
Jul 25 01:19:24.754991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:24.754949    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-jc4xw"
Jul 25 01:19:29.235963 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:29.235923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:29.287564 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:29.287528    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:31.188578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:31.188541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:31.243797 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:31.243765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:38.983135 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:38.983096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:39.077831 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.077794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:39.114128 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.114091    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjzn"]
Jul 25 01:19:39.114402 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.114390    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e527177c-0b8b-4395-a405-28c1c45bb364" containerName="discovery"
Jul 25 01:19:39.114446 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.114403    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e527177c-0b8b-4395-a405-28c1c45bb364" containerName="discovery"
Jul 25 01:19:39.114490 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.114458    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e527177c-0b8b-4395-a405-28c1c45bb364" containerName="discovery"
Jul 25 01:19:39.118275 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.118259    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn"
Jul 25 01:19:39.121428 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.121406    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Jul 25 01:19:39.122654 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.122630    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-6dtmp\""
Jul 25 01:19:39.122654 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.122640    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Jul 25 01:19:39.126139 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.125721    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjzn"]
Jul 25 01:19:39.159210 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.159177    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:39.172160 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.172135    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7clb5\" (UniqueName: \"kubernetes.io/projected/99779d22-1fb2-433e-ab58-7d649b3d073e-kube-api-access-7clb5\") pod \"kuadrant-operator-catalog-jgjzn\" (UID: \"99779d22-1fb2-433e-ab58-7d649b3d073e\") " pod="kuadrant-system/kuadrant-operator-catalog-jgjzn"
Jul 25 01:19:39.273594 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.273515    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7clb5\" (UniqueName: \"kubernetes.io/projected/99779d22-1fb2-433e-ab58-7d649b3d073e-kube-api-access-7clb5\") pod \"kuadrant-operator-catalog-jgjzn\" (UID: \"99779d22-1fb2-433e-ab58-7d649b3d073e\") " pod="kuadrant-system/kuadrant-operator-catalog-jgjzn"
Jul 25 01:19:39.281311 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.281289    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7clb5\" (UniqueName: \"kubernetes.io/projected/99779d22-1fb2-433e-ab58-7d649b3d073e-kube-api-access-7clb5\") pod \"kuadrant-operator-catalog-jgjzn\" (UID: \"99779d22-1fb2-433e-ab58-7d649b3d073e\") " pod="kuadrant-system/kuadrant-operator-catalog-jgjzn"
Jul 25 01:19:39.429070 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.429045    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn"
Jul 25 01:19:39.491282 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.491224    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjzn"]
Jul 25 01:19:39.555258 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.555238    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjzn"]
Jul 25 01:19:39.557291 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:19:39.557258    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99779d22_1fb2_433e_ab58_7d649b3d073e.slice/crio-f867f73ba22dd33fedbc264f79a2c39839162dbbff32df56ec3d64632b074558 WatchSource:0}: Error finding container f867f73ba22dd33fedbc264f79a2c39839162dbbff32df56ec3d64632b074558: Status 404 returned error can't find the container with id f867f73ba22dd33fedbc264f79a2c39839162dbbff32df56ec3d64632b074558
Jul 25 01:19:39.695118 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.695087    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hlxgt"]
Jul 25 01:19:39.697882 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.697866    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:39.705377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.705351    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hlxgt"]
Jul 25 01:19:39.777534 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.777505    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v9gl9\" (UniqueName: \"kubernetes.io/projected/59b701d4-aa2a-43c7-b164-baf5ff77989f-kube-api-access-v9gl9\") pod \"kuadrant-operator-catalog-hlxgt\" (UID: \"59b701d4-aa2a-43c7-b164-baf5ff77989f\") " pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:39.878100 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.878077    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v9gl9\" (UniqueName: \"kubernetes.io/projected/59b701d4-aa2a-43c7-b164-baf5ff77989f-kube-api-access-v9gl9\") pod \"kuadrant-operator-catalog-hlxgt\" (UID: \"59b701d4-aa2a-43c7-b164-baf5ff77989f\") " pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:39.886513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.886494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v9gl9\" (UniqueName: \"kubernetes.io/projected/59b701d4-aa2a-43c7-b164-baf5ff77989f-kube-api-access-v9gl9\") pod \"kuadrant-operator-catalog-hlxgt\" (UID: \"59b701d4-aa2a-43c7-b164-baf5ff77989f\") " pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:39.933506 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:39.933480    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn" event={"ID":"99779d22-1fb2-433e-ab58-7d649b3d073e","Type":"ContainerStarted","Data":"f867f73ba22dd33fedbc264f79a2c39839162dbbff32df56ec3d64632b074558"}
Jul 25 01:19:40.006916 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:40.006895    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:40.135181 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:40.135158    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hlxgt"]
Jul 25 01:19:40.137457 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:19:40.137427    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59b701d4_aa2a_43c7_b164_baf5ff77989f.slice/crio-3a01b67953ac4e350f92999dd1e254040f172e04b98034b0a12efeb852ed1cb4 WatchSource:0}: Error finding container 3a01b67953ac4e350f92999dd1e254040f172e04b98034b0a12efeb852ed1cb4: Status 404 returned error can't find the container with id 3a01b67953ac4e350f92999dd1e254040f172e04b98034b0a12efeb852ed1cb4
Jul 25 01:19:40.940677 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:40.940645    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt" event={"ID":"59b701d4-aa2a-43c7-b164-baf5ff77989f","Type":"ContainerStarted","Data":"3a01b67953ac4e350f92999dd1e254040f172e04b98034b0a12efeb852ed1cb4"}
Jul 25 01:19:41.945950 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:41.945912    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn" event={"ID":"99779d22-1fb2-433e-ab58-7d649b3d073e","Type":"ContainerStarted","Data":"6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523"}
Jul 25 01:19:41.946396 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:41.946000    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn" podUID="99779d22-1fb2-433e-ab58-7d649b3d073e" containerName="registry-server" containerID="cri-o://6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523" gracePeriod=2
Jul 25 01:19:41.947460 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:41.947426    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt" event={"ID":"59b701d4-aa2a-43c7-b164-baf5ff77989f","Type":"ContainerStarted","Data":"3b9aa7ac37834881f19b76a888301261c463583bdd2d5c6be3cee2e135cb262d"}
Jul 25 01:19:41.961506 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:41.961467    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn" podStartSLOduration=1.036867565 podStartE2EDuration="2.961454083s" totalImagesPullingTime="1.924586518s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:19:41.960567159 +0000 UTC m=+460.431523497" watchObservedRunningTime="2026-07-25 01:19:41.961454083 +0000 UTC m=+460.432410420"
Jul 25 01:19:41.975365 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:41.975331    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt" podStartSLOduration=1.6304219789999999 podStartE2EDuration="2.975317867s" totalImagesPullingTime="1.344895888s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:19:41.975064366 +0000 UTC m=+460.446020704" watchObservedRunningTime="2026-07-25 01:19:41.975317867 +0000 UTC m=+460.446274205"
Jul 25 01:19:42.189052 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.189030    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn"
Jul 25 01:19:42.296772 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.296708    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7clb5\" (UniqueName: \"kubernetes.io/projected/99779d22-1fb2-433e-ab58-7d649b3d073e-kube-api-access-7clb5\") pod \"99779d22-1fb2-433e-ab58-7d649b3d073e\" (UID: \"99779d22-1fb2-433e-ab58-7d649b3d073e\") "
Jul 25 01:19:42.298776 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.298753    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99779d22-1fb2-433e-ab58-7d649b3d073e-kube-api-access-7clb5" (OuterVolumeSpecName: "kube-api-access-7clb5") pod "99779d22-1fb2-433e-ab58-7d649b3d073e" (UID: "99779d22-1fb2-433e-ab58-7d649b3d073e"). InnerVolumeSpecName "kube-api-access-7clb5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:19:42.397155 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.397132    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7clb5\" (UniqueName: \"kubernetes.io/projected/99779d22-1fb2-433e-ab58-7d649b3d073e-kube-api-access-7clb5\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:19:42.952653 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.952623    2562 generic.go:356] "Generic (PLEG): container finished" podID="99779d22-1fb2-433e-ab58-7d649b3d073e" containerID="6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523" exitCode=0
Jul 25 01:19:42.953034 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.952672    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn"
Jul 25 01:19:42.953034 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.952713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn" event={"ID":"99779d22-1fb2-433e-ab58-7d649b3d073e","Type":"ContainerDied","Data":"6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523"}
Jul 25 01:19:42.953034 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.952765    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-jgjzn" event={"ID":"99779d22-1fb2-433e-ab58-7d649b3d073e","Type":"ContainerDied","Data":"f867f73ba22dd33fedbc264f79a2c39839162dbbff32df56ec3d64632b074558"}
Jul 25 01:19:42.953034 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.952791    2562 scope.go:117] "RemoveContainer" containerID="6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523"
Jul 25 01:19:42.963780 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.963746    2562 scope.go:117] "RemoveContainer" containerID="6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523"
Jul 25 01:19:42.964210 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:19:42.964181    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523\": container with ID starting with 6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523 not found: ID does not exist" containerID="6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523"
Jul 25 01:19:42.964321 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.964221    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523"} err="failed to get container status \"6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523\": rpc error: code = NotFound desc = could not find container \"6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523\": container with ID starting with 6bb47098e9272e37ed7d8a9a0b220122b249a593d39de15cb237fd95b0e4e523 not found: ID does not exist"
Jul 25 01:19:42.978070 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.978047    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjzn"]
Jul 25 01:19:42.981528 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:42.981503    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-jgjzn"]
Jul 25 01:19:44.075220 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:44.075187    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99779d22-1fb2-433e-ab58-7d649b3d073e" path="/var/lib/kubelet/pods/99779d22-1fb2-433e-ab58-7d649b3d073e/volumes"
Jul 25 01:19:50.007807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:50.007776    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:50.008448 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:50.007862    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:50.028718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:50.028699    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:51.005537 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:51.005512    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hlxgt"
Jul 25 01:19:51.695661 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:51.695622    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:19:51.797481 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:19:51.797432    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:02.379917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:02.379886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:02.441128 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:02.441099    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:02.498261 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:02.498216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:07.574919 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:07.574884    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:08.171818 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:08.171779    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:09.197816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:09.197778    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:11.687997 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.687953    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"]
Jul 25 01:20:11.688359 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.688324    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99779d22-1fb2-433e-ab58-7d649b3d073e" containerName="registry-server"
Jul 25 01:20:11.688359 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.688335    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="99779d22-1fb2-433e-ab58-7d649b3d073e" containerName="registry-server"
Jul 25 01:20:11.688432 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.688387    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="99779d22-1fb2-433e-ab58-7d649b3d073e" containerName="registry-server"
Jul 25 01:20:11.694574 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.694553    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"
Jul 25 01:20:11.699825 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.699805    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hbbds\""
Jul 25 01:20:11.705592 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.705562    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"]
Jul 25 01:20:11.816068 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.816035    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bztgq\" (UniqueName: \"kubernetes.io/projected/0381d3ae-a5f4-4465-a57e-d97a2300f0bc-kube-api-access-bztgq\") pod \"authorino-operator-75cc8d5cb7-fvgq4\" (UID: \"0381d3ae-a5f4-4465-a57e-d97a2300f0bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"
Jul 25 01:20:11.916525 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.916494    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bztgq\" (UniqueName: \"kubernetes.io/projected/0381d3ae-a5f4-4465-a57e-d97a2300f0bc-kube-api-access-bztgq\") pod \"authorino-operator-75cc8d5cb7-fvgq4\" (UID: \"0381d3ae-a5f4-4465-a57e-d97a2300f0bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"
Jul 25 01:20:11.925625 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:11.925595    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bztgq\" (UniqueName: \"kubernetes.io/projected/0381d3ae-a5f4-4465-a57e-d97a2300f0bc-kube-api-access-bztgq\") pod \"authorino-operator-75cc8d5cb7-fvgq4\" (UID: \"0381d3ae-a5f4-4465-a57e-d97a2300f0bc\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"
Jul 25 01:20:12.005184 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:12.005089    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"
Jul 25 01:20:12.139132 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:12.139109    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"]
Jul 25 01:20:12.141661 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:12.141633    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0381d3ae_a5f4_4465_a57e_d97a2300f0bc.slice/crio-277f73c85b0a2faebe0511d7d025930ebb2fd4e9be6d14474be45e38089c1af3 WatchSource:0}: Error finding container 277f73c85b0a2faebe0511d7d025930ebb2fd4e9be6d14474be45e38089c1af3: Status 404 returned error can't find the container with id 277f73c85b0a2faebe0511d7d025930ebb2fd4e9be6d14474be45e38089c1af3
Jul 25 01:20:12.795778 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:12.795742    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:13.074971 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:13.074886    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4" event={"ID":"0381d3ae-a5f4-4465-a57e-d97a2300f0bc","Type":"ContainerStarted","Data":"277f73c85b0a2faebe0511d7d025930ebb2fd4e9be6d14474be45e38089c1af3"}
Jul 25 01:20:14.866451 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:14.866366    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"]
Jul 25 01:20:14.869885 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:14.869867    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"
Jul 25 01:20:14.873317 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:14.873301    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-cbs7k\""
Jul 25 01:20:14.873317 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:14.873308    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Jul 25 01:20:14.889623 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:14.889597    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"]
Jul 25 01:20:14.943167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:14.943138    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwqgb\" (UniqueName: \"kubernetes.io/projected/68f1e9de-a0d2-4eb3-8d48-9bada87fa201-kube-api-access-dwqgb\") pod \"dns-operator-controller-manager-864dd78c4c-j2q5r\" (UID: \"68f1e9de-a0d2-4eb3-8d48-9bada87fa201\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"
Jul 25 01:20:15.044908 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.044874    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwqgb\" (UniqueName: \"kubernetes.io/projected/68f1e9de-a0d2-4eb3-8d48-9bada87fa201-kube-api-access-dwqgb\") pod \"dns-operator-controller-manager-864dd78c4c-j2q5r\" (UID: \"68f1e9de-a0d2-4eb3-8d48-9bada87fa201\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"
Jul 25 01:20:15.058744 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.058719    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwqgb\" (UniqueName: \"kubernetes.io/projected/68f1e9de-a0d2-4eb3-8d48-9bada87fa201-kube-api-access-dwqgb\") pod \"dns-operator-controller-manager-864dd78c4c-j2q5r\" (UID: \"68f1e9de-a0d2-4eb3-8d48-9bada87fa201\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"
Jul 25 01:20:15.084941 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.084900    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4" event={"ID":"0381d3ae-a5f4-4465-a57e-d97a2300f0bc","Type":"ContainerStarted","Data":"d5e9fe6f3b456972bce7cd893353555327d7c11e5fa3b91007e7200eecbf2051"}
Jul 25 01:20:15.085137 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.085037    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"
Jul 25 01:20:15.104095 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.104043    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4" podStartSLOduration=1.855844772 podStartE2EDuration="4.104023993s" totalImagesPullingTime="2.248179221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:15.101973469 +0000 UTC m=+493.572929807" watchObservedRunningTime="2026-07-25 01:20:15.104023993 +0000 UTC m=+493.574980332"
Jul 25 01:20:15.179257 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.179227    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"
Jul 25 01:20:15.306654 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.306633    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"]
Jul 25 01:20:15.308815 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:15.308784    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68f1e9de_a0d2_4eb3_8d48_9bada87fa201.slice/crio-4c835c5feee0d5c58c9a1083e73b5a5a3a7cea486649dc21aa670b9130e7b35c WatchSource:0}: Error finding container 4c835c5feee0d5c58c9a1083e73b5a5a3a7cea486649dc21aa670b9130e7b35c: Status 404 returned error can't find the container with id 4c835c5feee0d5c58c9a1083e73b5a5a3a7cea486649dc21aa670b9130e7b35c
Jul 25 01:20:15.585384 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.585313    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:15.891377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:15.891342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:16.089620 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:16.089574    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r" event={"ID":"68f1e9de-a0d2-4eb3-8d48-9bada87fa201","Type":"ContainerStarted","Data":"4c835c5feee0d5c58c9a1083e73b5a5a3a7cea486649dc21aa670b9130e7b35c"}
Jul 25 01:20:17.095322 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.095292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r" event={"ID":"68f1e9de-a0d2-4eb3-8d48-9bada87fa201","Type":"ContainerStarted","Data":"a4fefa9f37ecffd969fa42a32de4202a17318d9f04af6122b1e9ade23de50a61"}
Jul 25 01:20:17.095647 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.095460    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"
Jul 25 01:20:17.116184 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.116144    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r" podStartSLOduration=1.401500802 podStartE2EDuration="3.116130483s" totalImagesPullingTime="1.714629681s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:17.113645579 +0000 UTC m=+495.584601918" watchObservedRunningTime="2026-07-25 01:20:17.116130483 +0000 UTC m=+495.587086821"
Jul 25 01:20:17.564610 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.564579    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"]
Jul 25 01:20:17.567937 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.567915    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:17.570684 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.570664    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-c4b5v\""
Jul 25 01:20:17.585335 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.585314    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"]
Jul 25 01:20:17.666743 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.666721    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qglmq\" (UniqueName: \"kubernetes.io/projected/2c84080b-8632-4d0d-8706-c9290a171a60-kube-api-access-qglmq\") pod \"limitador-operator-controller-manager-859d7b78bd-xcjjs\" (UID: \"2c84080b-8632-4d0d-8706-c9290a171a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:17.767196 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.767175    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qglmq\" (UniqueName: \"kubernetes.io/projected/2c84080b-8632-4d0d-8706-c9290a171a60-kube-api-access-qglmq\") pod \"limitador-operator-controller-manager-859d7b78bd-xcjjs\" (UID: \"2c84080b-8632-4d0d-8706-c9290a171a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:17.777490 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.777461    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qglmq\" (UniqueName: \"kubernetes.io/projected/2c84080b-8632-4d0d-8706-c9290a171a60-kube-api-access-qglmq\") pod \"limitador-operator-controller-manager-859d7b78bd-xcjjs\" (UID: \"2c84080b-8632-4d0d-8706-c9290a171a60\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:17.877302 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:17.877280    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:18.008797 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:18.008773    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"]
Jul 25 01:20:18.010946 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:18.010918    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c84080b_8632_4d0d_8706_c9290a171a60.slice/crio-9ef2f2c14ba0d1f447c2dbd92f0e0edd1df7192876b7d279f4db9b4744ee6af1 WatchSource:0}: Error finding container 9ef2f2c14ba0d1f447c2dbd92f0e0edd1df7192876b7d279f4db9b4744ee6af1: Status 404 returned error can't find the container with id 9ef2f2c14ba0d1f447c2dbd92f0e0edd1df7192876b7d279f4db9b4744ee6af1
Jul 25 01:20:18.012185 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:18.012146    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:18.100193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:18.100161    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs" event={"ID":"2c84080b-8632-4d0d-8706-c9290a171a60","Type":"ContainerStarted","Data":"9ef2f2c14ba0d1f447c2dbd92f0e0edd1df7192876b7d279f4db9b4744ee6af1"}
Jul 25 01:20:18.174175 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:18.174083    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:20.110520 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:20.110486    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs" event={"ID":"2c84080b-8632-4d0d-8706-c9290a171a60","Type":"ContainerStarted","Data":"7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948"}
Jul 25 01:20:20.110845 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:20.110617    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:20.128532 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:20.128489    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs" podStartSLOduration=1.737433156 podStartE2EDuration="3.128475378s" totalImagesPullingTime="1.391042222s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:20.127122159 +0000 UTC m=+498.598078498" watchObservedRunningTime="2026-07-25 01:20:20.128475378 +0000 UTC m=+498.599431716"
Jul 25 01:20:20.650752 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:20.650705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:21.816826 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:21.816792    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"]
Jul 25 01:20:21.820193 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:21.820175    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:21.823146 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:21.823127    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sjbxw\""
Jul 25 01:20:21.834740 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:21.834721    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"]
Jul 25 01:20:21.901891 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:21.901864    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jm2jj\" (UniqueName: \"kubernetes.io/projected/a3518d4b-c65c-4336-b77e-d421964a9b4d-kube-api-access-jm2jj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" (UID: \"a3518d4b-c65c-4336-b77e-d421964a9b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:21.902051 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:21.901929    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3518d4b-c65c-4336-b77e-d421964a9b4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" (UID: \"a3518d4b-c65c-4336-b77e-d421964a9b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:22.002828 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:22.002803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3518d4b-c65c-4336-b77e-d421964a9b4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" (UID: \"a3518d4b-c65c-4336-b77e-d421964a9b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:22.002936 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:22.002845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jm2jj\" (UniqueName: \"kubernetes.io/projected/a3518d4b-c65c-4336-b77e-d421964a9b4d-kube-api-access-jm2jj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" (UID: \"a3518d4b-c65c-4336-b77e-d421964a9b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:22.003189 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:22.003171    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3518d4b-c65c-4336-b77e-d421964a9b4d-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" (UID: \"a3518d4b-c65c-4336-b77e-d421964a9b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:22.010816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:22.010790    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jm2jj\" (UniqueName: \"kubernetes.io/projected/a3518d4b-c65c-4336-b77e-d421964a9b4d-kube-api-access-jm2jj\") pod \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" (UID: \"a3518d4b-c65c-4336-b77e-d421964a9b4d\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:22.129402 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:22.129370    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:22.262526 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:22.262504    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"]
Jul 25 01:20:22.264296 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:22.264273    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3518d4b_c65c_4336_b77e_d421964a9b4d.slice/crio-7018a1f34883e1131ed160d68c523affdfa1b72aafb0deca7172bddf5273681c WatchSource:0}: Error finding container 7018a1f34883e1131ed160d68c523affdfa1b72aafb0deca7172bddf5273681c: Status 404 returned error can't find the container with id 7018a1f34883e1131ed160d68c523affdfa1b72aafb0deca7172bddf5273681c
Jul 25 01:20:23.126912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:23.126876    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp" event={"ID":"a3518d4b-c65c-4336-b77e-d421964a9b4d","Type":"ContainerStarted","Data":"7018a1f34883e1131ed160d68c523affdfa1b72aafb0deca7172bddf5273681c"}
Jul 25 01:20:23.187691 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:23.187654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:23.289437 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:23.289396    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:23.388185 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:23.388101    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:23.488384 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:23.488348    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:26.092807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:26.092766    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-fvgq4"
Jul 25 01:20:26.998727 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:26.998697    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:27.096711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:27.096679    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:27.146174 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:27.146143    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp" event={"ID":"a3518d4b-c65c-4336-b77e-d421964a9b4d","Type":"ContainerStarted","Data":"c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1"}
Jul 25 01:20:27.146295 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:27.146278    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:27.168369 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:27.168326    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp" podStartSLOduration=1.478681278 podStartE2EDuration="6.168310295s" totalImagesPullingTime="4.689629017s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:27.165672639 +0000 UTC m=+505.636628977" watchObservedRunningTime="2026-07-25 01:20:27.168310295 +0000 UTC m=+505.639266632"
Jul 25 01:20:28.102723 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:28.102694    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-j2q5r"
Jul 25 01:20:29.557501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:29.557468    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:29.731706 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:29.731654    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:31.020286 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:31.020259    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:31.117033 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:31.117009    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:31.152869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:31.152838    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:34.511483 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:34.511443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:34.612422 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:34.612389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:38.154053 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:38.154023    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:39.216633 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.216595    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"]
Jul 25 01:20:39.220061 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.220045    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:39.231409 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.231382    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"]
Jul 25 01:20:39.239101 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.239070    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d956a937-9494-477e-9840-85f62424293a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" (UID: \"d956a937-9494-477e-9840-85f62424293a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:39.239247 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.239135    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mflgh\" (UniqueName: \"kubernetes.io/projected/d956a937-9494-477e-9840-85f62424293a-kube-api-access-mflgh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" (UID: \"d956a937-9494-477e-9840-85f62424293a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:39.339779 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.339751    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d956a937-9494-477e-9840-85f62424293a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" (UID: \"d956a937-9494-477e-9840-85f62424293a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:39.339922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.339792    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mflgh\" (UniqueName: \"kubernetes.io/projected/d956a937-9494-477e-9840-85f62424293a-kube-api-access-mflgh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" (UID: \"d956a937-9494-477e-9840-85f62424293a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:39.340231 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.340199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d956a937-9494-477e-9840-85f62424293a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" (UID: \"d956a937-9494-477e-9840-85f62424293a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:39.355792 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.355770    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mflgh\" (UniqueName: \"kubernetes.io/projected/d956a937-9494-477e-9840-85f62424293a-kube-api-access-mflgh\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" (UID: \"d956a937-9494-477e-9840-85f62424293a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:39.530732 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.530656    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:39.658736 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.658613    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"]
Jul 25 01:20:39.661152 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:39.661130    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd956a937_9494_477e_9840_85f62424293a.slice/crio-664eb681b0f731028e2d9725867811874bf12fd441572f2ca499006714f9dba4 WatchSource:0}: Error finding container 664eb681b0f731028e2d9725867811874bf12fd441572f2ca499006714f9dba4: Status 404 returned error can't find the container with id 664eb681b0f731028e2d9725867811874bf12fd441572f2ca499006714f9dba4
Jul 25 01:20:39.874173 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.874139    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"]
Jul 25 01:20:39.882694 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.882669    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"]
Jul 25 01:20:39.900222 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.900191    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"]
Jul 25 01:20:39.900442 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.900403    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp" podUID="a3518d4b-c65c-4336-b77e-d421964a9b4d" containerName="manager" containerID="cri-o://c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1" gracePeriod=2
Jul 25 01:20:39.901635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.901617    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"]
Jul 25 01:20:39.901964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.901951    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d956a937-9494-477e-9840-85f62424293a" containerName="manager"
Jul 25 01:20:39.902025 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.901968    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d956a937-9494-477e-9840-85f62424293a" containerName="manager"
Jul 25 01:20:39.902062 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.902054    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d956a937-9494-477e-9840-85f62424293a" containerName="manager"
Jul 25 01:20:39.904915 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.904899    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:39.909771 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.909714    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"]
Jul 25 01:20:39.915124 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:39.915094    2562 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"a3518d4b-c65c-4336-b77e-d421964a9b4d\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\": pods \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:39.917262 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.917240    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"]
Jul 25 01:20:39.927614 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.926844    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"]
Jul 25 01:20:39.927614 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.927084    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs" podUID="2c84080b-8632-4d0d-8706-c9290a171a60" containerName="manager" containerID="cri-o://7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948" gracePeriod=2
Jul 25 01:20:39.928454 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.928434    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"]
Jul 25 01:20:39.928758 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.928746    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3518d4b-c65c-4336-b77e-d421964a9b4d" containerName="manager"
Jul 25 01:20:39.928800 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.928762    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3518d4b-c65c-4336-b77e-d421964a9b4d" containerName="manager"
Jul 25 01:20:39.928834 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.928815    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a3518d4b-c65c-4336-b77e-d421964a9b4d" containerName="manager"
Jul 25 01:20:39.931806 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.931787    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:39.933559 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.933537    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"]
Jul 25 01:20:39.947601 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:39.947314    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="a3518d4b-c65c-4336-b77e-d421964a9b4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:39.947751 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.947687    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2fwc\" (UniqueName: \"kubernetes.io/projected/10b45418-63b4-4707-b269-66c39aa5f6ae-kube-api-access-l2fwc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7drc\" (UID: \"10b45418-63b4-4707-b269-66c39aa5f6ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:39.947812 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.947772    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1d4f597-85d2-4f34-ade8-8fe40589a06f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7xkcb\" (UID: \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:39.947889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.947870    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b45418-63b4-4707-b269-66c39aa5f6ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7drc\" (UID: \"10b45418-63b4-4707-b269-66c39aa5f6ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:39.947968 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.947904    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9vbq\" (UniqueName: \"kubernetes.io/projected/e1d4f597-85d2-4f34-ade8-8fe40589a06f-kube-api-access-s9vbq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7xkcb\" (UID: \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:39.949897 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:39.949864    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xcjjs\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="2c84080b-8632-4d0d-8706-c9290a171a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:39.950142 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.950124    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"]
Jul 25 01:20:39.951313 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.951296    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"]
Jul 25 01:20:39.951598 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.951587    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c84080b-8632-4d0d-8706-c9290a171a60" containerName="manager"
Jul 25 01:20:39.951634 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.951600    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c84080b-8632-4d0d-8706-c9290a171a60" containerName="manager"
Jul 25 01:20:39.951670 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.951665    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c84080b-8632-4d0d-8706-c9290a171a60" containerName="manager"
Jul 25 01:20:39.952274 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:39.952248    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="a3518d4b-c65c-4336-b77e-d421964a9b4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:39.961549 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.961532    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"
Jul 25 01:20:39.965291 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:39.965273    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"]
Jul 25 01:20:39.975525 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:39.975501    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-xcjjs\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="2c84080b-8632-4d0d-8706-c9290a171a60" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:39.977717 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:39.977695    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-g9jcp\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="a3518d4b-c65c-4336-b77e-d421964a9b4d" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:40.048675 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.048645    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7t6v2\" (UniqueName: \"kubernetes.io/projected/d1dea10f-e012-4dc5-b06c-df4d4a1ccb86-kube-api-access-7t6v2\") pod \"limitador-operator-controller-manager-859d7b78bd-m4qb8\" (UID: \"d1dea10f-e012-4dc5-b06c-df4d4a1ccb86\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"
Jul 25 01:20:40.048789 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.048727    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b45418-63b4-4707-b269-66c39aa5f6ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7drc\" (UID: \"10b45418-63b4-4707-b269-66c39aa5f6ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:40.048789 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.048754    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s9vbq\" (UniqueName: \"kubernetes.io/projected/e1d4f597-85d2-4f34-ade8-8fe40589a06f-kube-api-access-s9vbq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7xkcb\" (UID: \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:40.048789 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.048785    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2fwc\" (UniqueName: \"kubernetes.io/projected/10b45418-63b4-4707-b269-66c39aa5f6ae-kube-api-access-l2fwc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7drc\" (UID: \"10b45418-63b4-4707-b269-66c39aa5f6ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:40.048958 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.048815    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1d4f597-85d2-4f34-ade8-8fe40589a06f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7xkcb\" (UID: \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:40.049223 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.049193    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1d4f597-85d2-4f34-ade8-8fe40589a06f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7xkcb\" (UID: \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:40.049616 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.049599    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b45418-63b4-4707-b269-66c39aa5f6ae-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7drc\" (UID: \"10b45418-63b4-4707-b269-66c39aa5f6ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:40.058362 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.058341    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2fwc\" (UniqueName: \"kubernetes.io/projected/10b45418-63b4-4707-b269-66c39aa5f6ae-kube-api-access-l2fwc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-n7drc\" (UID: \"10b45418-63b4-4707-b269-66c39aa5f6ae\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:40.059158 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.059138    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9vbq\" (UniqueName: \"kubernetes.io/projected/e1d4f597-85d2-4f34-ade8-8fe40589a06f-kube-api-access-s9vbq\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-7xkcb\" (UID: \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:40.150012 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.149916    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7t6v2\" (UniqueName: \"kubernetes.io/projected/d1dea10f-e012-4dc5-b06c-df4d4a1ccb86-kube-api-access-7t6v2\") pod \"limitador-operator-controller-manager-859d7b78bd-m4qb8\" (UID: \"d1dea10f-e012-4dc5-b06c-df4d4a1ccb86\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"
Jul 25 01:20:40.158744 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.158724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7t6v2\" (UniqueName: \"kubernetes.io/projected/d1dea10f-e012-4dc5-b06c-df4d4a1ccb86-kube-api-access-7t6v2\") pod \"limitador-operator-controller-manager-859d7b78bd-m4qb8\" (UID: \"d1dea10f-e012-4dc5-b06c-df4d4a1ccb86\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"
Jul 25 01:20:40.159464 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.159438    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:40.161461 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.161446    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:40.205107 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:40.204339    2562 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t" container="manager"
Jul 25 01:20:40.207939 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.207915    2562 generic.go:356] "Generic (PLEG): container finished" podID="2c84080b-8632-4d0d-8706-c9290a171a60" containerID="7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948" exitCode=0
Jul 25 01:20:40.208055 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.207951    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-xcjjs"
Jul 25 01:20:40.208055 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.208018    2562 scope.go:117] "RemoveContainer" containerID="7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948"
Jul 25 01:20:40.209549 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.209519    2562 generic.go:356] "Generic (PLEG): container finished" podID="a3518d4b-c65c-4336-b77e-d421964a9b4d" containerID="c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1" exitCode=0
Jul 25 01:20:40.209549 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.209544    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-g9jcp"
Jul 25 01:20:40.217386 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.217372    2562 scope.go:117] "RemoveContainer" containerID="7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948"
Jul 25 01:20:40.217729 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:40.217629    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948\": container with ID starting with 7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948 not found: ID does not exist" containerID="7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948"
Jul 25 01:20:40.217862 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.217832    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948"} err="failed to get container status \"7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948\": rpc error: code = NotFound desc = could not find container \"7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948\": container with ID starting with 7706514dfd85292c6639f9c38d97b5561cbac3bf99631bb2d7256a9c4c39a948 not found: ID does not exist"
Jul 25 01:20:40.217973 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.217865    2562 scope.go:117] "RemoveContainer" containerID="c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1"
Jul 25 01:20:40.226085 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.226071    2562 scope.go:117] "RemoveContainer" containerID="c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1"
Jul 25 01:20:40.226338 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:40.226321    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1\": container with ID starting with c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1 not found: ID does not exist" containerID="c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1"
Jul 25 01:20:40.226391 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.226343    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1"} err="failed to get container status \"c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1\": rpc error: code = NotFound desc = could not find container \"c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1\": container with ID starting with c9b362cb0fcc2861a97c2abf63dba4c661afc29bbc68829e66f5a5f6da3fb0f1 not found: ID does not exist"
Jul 25 01:20:40.250720 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.250700    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qglmq\" (UniqueName: \"kubernetes.io/projected/2c84080b-8632-4d0d-8706-c9290a171a60-kube-api-access-qglmq\") pod \"2c84080b-8632-4d0d-8706-c9290a171a60\" (UID: \"2c84080b-8632-4d0d-8706-c9290a171a60\") "
Jul 25 01:20:40.250813 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.250731    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3518d4b-c65c-4336-b77e-d421964a9b4d-extensions-socket-volume\") pod \"a3518d4b-c65c-4336-b77e-d421964a9b4d\" (UID: \"a3518d4b-c65c-4336-b77e-d421964a9b4d\") "
Jul 25 01:20:40.250813 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.250759    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jm2jj\" (UniqueName: \"kubernetes.io/projected/a3518d4b-c65c-4336-b77e-d421964a9b4d-kube-api-access-jm2jj\") pod \"a3518d4b-c65c-4336-b77e-d421964a9b4d\" (UID: \"a3518d4b-c65c-4336-b77e-d421964a9b4d\") "
Jul 25 01:20:40.251198 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.251175    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3518d4b-c65c-4336-b77e-d421964a9b4d-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "a3518d4b-c65c-4336-b77e-d421964a9b4d" (UID: "a3518d4b-c65c-4336-b77e-d421964a9b4d"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:20:40.252686 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.252661    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3518d4b-c65c-4336-b77e-d421964a9b4d-kube-api-access-jm2jj" (OuterVolumeSpecName: "kube-api-access-jm2jj") pod "a3518d4b-c65c-4336-b77e-d421964a9b4d" (UID: "a3518d4b-c65c-4336-b77e-d421964a9b4d"). InnerVolumeSpecName "kube-api-access-jm2jj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:40.252760 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.252666    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c84080b-8632-4d0d-8706-c9290a171a60-kube-api-access-qglmq" (OuterVolumeSpecName: "kube-api-access-qglmq") pod "2c84080b-8632-4d0d-8706-c9290a171a60" (UID: "2c84080b-8632-4d0d-8706-c9290a171a60"). InnerVolumeSpecName "kube-api-access-qglmq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:40.340049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.340024    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:40.350959 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.350938    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:40.351307 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.351289    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jm2jj\" (UniqueName: \"kubernetes.io/projected/a3518d4b-c65c-4336-b77e-d421964a9b4d-kube-api-access-jm2jj\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:20:40.351376 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.351312    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qglmq\" (UniqueName: \"kubernetes.io/projected/2c84080b-8632-4d0d-8706-c9290a171a60-kube-api-access-qglmq\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:20:40.351376 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.351325    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3518d4b-c65c-4336-b77e-d421964a9b4d-extensions-socket-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:20:40.357930 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.357887    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"
Jul 25 01:20:40.511717 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.511442    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"]
Jul 25 01:20:40.515293 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:40.515247    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d4f597_85d2_4f34_ade8_8fe40589a06f.slice/crio-9c16a76c1b26271a77f15998edbceee9b8e10bddfc84eed1a06b73fad16d6992 WatchSource:0}: Error finding container 9c16a76c1b26271a77f15998edbceee9b8e10bddfc84eed1a06b73fad16d6992: Status 404 returned error can't find the container with id 9c16a76c1b26271a77f15998edbceee9b8e10bddfc84eed1a06b73fad16d6992
Jul 25 01:20:40.531689 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.531628    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"]
Jul 25 01:20:40.535318 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:40.534965    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10b45418_63b4_4707_b269_66c39aa5f6ae.slice/crio-4b9049437a2f28db18d6f5c64f8b6ca9ec2a88e473b21d3dd805246817e4cb30 WatchSource:0}: Error finding container 4b9049437a2f28db18d6f5c64f8b6ca9ec2a88e473b21d3dd805246817e4cb30: Status 404 returned error can't find the container with id 4b9049437a2f28db18d6f5c64f8b6ca9ec2a88e473b21d3dd805246817e4cb30
Jul 25 01:20:40.554123 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:40.554106    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"]
Jul 25 01:20:40.559900 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:40.559873    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1dea10f_e012_4dc5_b06c_df4d4a1ccb86.slice/crio-17320cdb185791cb9de55a5defbc5df28db65e4c92b3f5a5f487ee43fd4c0165 WatchSource:0}: Error finding container 17320cdb185791cb9de55a5defbc5df28db65e4c92b3f5a5f487ee43fd4c0165: Status 404 returned error can't find the container with id 17320cdb185791cb9de55a5defbc5df28db65e4c92b3f5a5f487ee43fd4c0165
Jul 25 01:20:41.216413 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.216377    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8" event={"ID":"d1dea10f-e012-4dc5-b06c-df4d4a1ccb86","Type":"ContainerStarted","Data":"a0da217697a209a089fed56c6244b5d253bcca72cc17666545d05a208a3db3b7"}
Jul 25 01:20:41.216580 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.216458    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8" event={"ID":"d1dea10f-e012-4dc5-b06c-df4d4a1ccb86","Type":"ContainerStarted","Data":"17320cdb185791cb9de55a5defbc5df28db65e4c92b3f5a5f487ee43fd4c0165"}
Jul 25 01:20:41.216580 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.216481    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"
Jul 25 01:20:41.218068 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.217965    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc" event={"ID":"10b45418-63b4-4707-b269-66c39aa5f6ae","Type":"ContainerStarted","Data":"52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419"}
Jul 25 01:20:41.218068 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.218026    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc" event={"ID":"10b45418-63b4-4707-b269-66c39aa5f6ae","Type":"ContainerStarted","Data":"4b9049437a2f28db18d6f5c64f8b6ca9ec2a88e473b21d3dd805246817e4cb30"}
Jul 25 01:20:41.218468 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.218121    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:41.220471 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.220447    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb" event={"ID":"e1d4f597-85d2-4f34-ade8-8fe40589a06f","Type":"ContainerStarted","Data":"43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab"}
Jul 25 01:20:41.220595 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.220515    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:41.220595 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.220553    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb" event={"ID":"e1d4f597-85d2-4f34-ade8-8fe40589a06f","Type":"ContainerStarted","Data":"9c16a76c1b26271a77f15998edbceee9b8e10bddfc84eed1a06b73fad16d6992"}
Jul 25 01:20:41.222090 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.222061    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t" podUID="d956a937-9494-477e-9840-85f62424293a" containerName="manager" containerID="cri-o://b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75" gracePeriod=2
Jul 25 01:20:41.235395 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.235355    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8" podStartSLOduration=2.235342184 podStartE2EDuration="2.235342184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:41.233675838 +0000 UTC m=+519.704632174" watchObservedRunningTime="2026-07-25 01:20:41.235342184 +0000 UTC m=+519.706298500"
Jul 25 01:20:41.252378 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.252337    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb" podStartSLOduration=2.25232396 podStartE2EDuration="2.25232396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:41.250873205 +0000 UTC m=+519.721829543" watchObservedRunningTime="2026-07-25 01:20:41.25232396 +0000 UTC m=+519.723280298"
Jul 25 01:20:41.253203 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:41.253179    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="d956a937-9494-477e-9840-85f62424293a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:41.274704 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.274660    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc" podStartSLOduration=2.274647097 podStartE2EDuration="2.274647097s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:41.273674259 +0000 UTC m=+519.744630600" watchObservedRunningTime="2026-07-25 01:20:41.274647097 +0000 UTC m=+519.745603435"
Jul 25 01:20:41.463514 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.463489    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:41.465801 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:41.465777    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="d956a937-9494-477e-9840-85f62424293a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:41.563222 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.563149    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mflgh\" (UniqueName: \"kubernetes.io/projected/d956a937-9494-477e-9840-85f62424293a-kube-api-access-mflgh\") pod \"d956a937-9494-477e-9840-85f62424293a\" (UID: \"d956a937-9494-477e-9840-85f62424293a\") "
Jul 25 01:20:41.563222 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.563210    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d956a937-9494-477e-9840-85f62424293a-extensions-socket-volume\") pod \"d956a937-9494-477e-9840-85f62424293a\" (UID: \"d956a937-9494-477e-9840-85f62424293a\") "
Jul 25 01:20:41.563574 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.563550    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d956a937-9494-477e-9840-85f62424293a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "d956a937-9494-477e-9840-85f62424293a" (UID: "d956a937-9494-477e-9840-85f62424293a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:20:41.565154 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.565137    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d956a937-9494-477e-9840-85f62424293a-kube-api-access-mflgh" (OuterVolumeSpecName: "kube-api-access-mflgh") pod "d956a937-9494-477e-9840-85f62424293a" (UID: "d956a937-9494-477e-9840-85f62424293a"). InnerVolumeSpecName "kube-api-access-mflgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:41.664358 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.664332    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mflgh\" (UniqueName: \"kubernetes.io/projected/d956a937-9494-477e-9840-85f62424293a-kube-api-access-mflgh\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:20:41.664358 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:41.664356    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/d956a937-9494-477e-9840-85f62424293a-extensions-socket-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:20:42.073738 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:42.073705    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="d956a937-9494-477e-9840-85f62424293a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:42.074591 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.074563    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c84080b-8632-4d0d-8706-c9290a171a60" path="/var/lib/kubelet/pods/2c84080b-8632-4d0d-8706-c9290a171a60/volumes"
Jul 25 01:20:42.074965 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.074947    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3518d4b-c65c-4336-b77e-d421964a9b4d" path="/var/lib/kubelet/pods/a3518d4b-c65c-4336-b77e-d421964a9b4d/volumes"
Jul 25 01:20:42.075337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.075320    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d956a937-9494-477e-9840-85f62424293a" path="/var/lib/kubelet/pods/d956a937-9494-477e-9840-85f62424293a/volumes"
Jul 25 01:20:42.228738 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.228714    2562 generic.go:356] "Generic (PLEG): container finished" podID="d956a937-9494-477e-9840-85f62424293a" containerID="b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75" exitCode=0
Jul 25 01:20:42.229109 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.228758    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:42.229109 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.228852    2562 scope.go:117] "RemoveContainer" containerID="b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75"
Jul 25 01:20:42.233537 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:42.233515    2562 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-znz2t\" is forbidden: User \"system:node:ip-10-0-129-103.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-129-103.ec2.internal' and this object" podUID="d956a937-9494-477e-9840-85f62424293a" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-znz2t"
Jul 25 01:20:42.236736 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.236721    2562 scope.go:117] "RemoveContainer" containerID="b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75"
Jul 25 01:20:42.236995 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:42.236965    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75\": container with ID starting with b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75 not found: ID does not exist" containerID="b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75"
Jul 25 01:20:42.237038 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.237006    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75"} err="failed to get container status \"b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75\": rpc error: code = NotFound desc = could not find container \"b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75\": container with ID starting with b6c347906b03d5ea1cf25465bf0e9f6aed4568498a5581c7d91d0547f715be75 not found: ID does not exist"
Jul 25 01:20:42.541567 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.541535    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:42.610844 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:42.610816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:43.142795 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:43.142765    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:43.249081 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:43.249041    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:44.840923 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:44.840886    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:46.143738 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:46.143698    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:46.243857 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:46.243824    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:50.741795 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:50.741761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:50.843159 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:50.843127    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:52.231386 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.231355    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:52.231722 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.231435    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-m4qb8"
Jul 25 01:20:52.231722 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.231480    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:20:52.311774 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.311741    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"]
Jul 25 01:20:52.311931 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.311907    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc" podUID="10b45418-63b4-4707-b269-66c39aa5f6ae" containerName="manager" containerID="cri-o://52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419" gracePeriod=10
Jul 25 01:20:52.554813 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.554790    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:52.650119 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.650091    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b45418-63b4-4707-b269-66c39aa5f6ae-extensions-socket-volume\") pod \"10b45418-63b4-4707-b269-66c39aa5f6ae\" (UID: \"10b45418-63b4-4707-b269-66c39aa5f6ae\") "
Jul 25 01:20:52.650252 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.650154    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l2fwc\" (UniqueName: \"kubernetes.io/projected/10b45418-63b4-4707-b269-66c39aa5f6ae-kube-api-access-l2fwc\") pod \"10b45418-63b4-4707-b269-66c39aa5f6ae\" (UID: \"10b45418-63b4-4707-b269-66c39aa5f6ae\") "
Jul 25 01:20:52.650447 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.650425    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10b45418-63b4-4707-b269-66c39aa5f6ae-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "10b45418-63b4-4707-b269-66c39aa5f6ae" (UID: "10b45418-63b4-4707-b269-66c39aa5f6ae"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:20:52.652102 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.652080    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10b45418-63b4-4707-b269-66c39aa5f6ae-kube-api-access-l2fwc" (OuterVolumeSpecName: "kube-api-access-l2fwc") pod "10b45418-63b4-4707-b269-66c39aa5f6ae" (UID: "10b45418-63b4-4707-b269-66c39aa5f6ae"). InnerVolumeSpecName "kube-api-access-l2fwc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:20:52.750627 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.750602    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/10b45418-63b4-4707-b269-66c39aa5f6ae-extensions-socket-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:20:52.750627 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.750623    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l2fwc\" (UniqueName: \"kubernetes.io/projected/10b45418-63b4-4707-b269-66c39aa5f6ae-kube-api-access-l2fwc\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:20:52.764947 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.764919    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"]
Jul 25 01:20:52.765268 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.765255    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10b45418-63b4-4707-b269-66c39aa5f6ae" containerName="manager"
Jul 25 01:20:52.765268 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.765268    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="10b45418-63b4-4707-b269-66c39aa5f6ae" containerName="manager"
Jul 25 01:20:52.765352 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.765318    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="10b45418-63b4-4707-b269-66c39aa5f6ae" containerName="manager"
Jul 25 01:20:52.767247 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.767231    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:52.781539 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.781520    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"]
Jul 25 01:20:52.851581 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.851529    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldgqx\" (UniqueName: \"kubernetes.io/projected/f9afee68-ba03-440b-a6fd-067287a274f0-kube-api-access-ldgqx\") pod \"kuadrant-operator-controller-manager-86878c769f-qhvjf\" (UID: \"f9afee68-ba03-440b-a6fd-067287a274f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:52.851581 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.851568    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9afee68-ba03-440b-a6fd-067287a274f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qhvjf\" (UID: \"f9afee68-ba03-440b-a6fd-067287a274f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:52.952768 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.952743    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldgqx\" (UniqueName: \"kubernetes.io/projected/f9afee68-ba03-440b-a6fd-067287a274f0-kube-api-access-ldgqx\") pod \"kuadrant-operator-controller-manager-86878c769f-qhvjf\" (UID: \"f9afee68-ba03-440b-a6fd-067287a274f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:52.952844 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.952781    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9afee68-ba03-440b-a6fd-067287a274f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qhvjf\" (UID: \"f9afee68-ba03-440b-a6fd-067287a274f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:52.953146 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.953130    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9afee68-ba03-440b-a6fd-067287a274f0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qhvjf\" (UID: \"f9afee68-ba03-440b-a6fd-067287a274f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:52.961222 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:52.961199    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldgqx\" (UniqueName: \"kubernetes.io/projected/f9afee68-ba03-440b-a6fd-067287a274f0-kube-api-access-ldgqx\") pod \"kuadrant-operator-controller-manager-86878c769f-qhvjf\" (UID: \"f9afee68-ba03-440b-a6fd-067287a274f0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:53.077049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.077029    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:53.210516 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.210486    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"]
Jul 25 01:20:53.212663 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:20:53.212634    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9afee68_ba03_440b_a6fd_067287a274f0.slice/crio-ac8cd3f906577f6fe643ee11b43dab1d17ffc27855a9391c23633eb855d00add WatchSource:0}: Error finding container ac8cd3f906577f6fe643ee11b43dab1d17ffc27855a9391c23633eb855d00add: Status 404 returned error can't find the container with id ac8cd3f906577f6fe643ee11b43dab1d17ffc27855a9391c23633eb855d00add
Jul 25 01:20:53.281879 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.281853    2562 generic.go:356] "Generic (PLEG): container finished" podID="10b45418-63b4-4707-b269-66c39aa5f6ae" containerID="52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419" exitCode=0
Jul 25 01:20:53.282230 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.281891    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"
Jul 25 01:20:53.282230 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.281931    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc" event={"ID":"10b45418-63b4-4707-b269-66c39aa5f6ae","Type":"ContainerDied","Data":"52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419"}
Jul 25 01:20:53.282230 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.281996    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc" event={"ID":"10b45418-63b4-4707-b269-66c39aa5f6ae","Type":"ContainerDied","Data":"4b9049437a2f28db18d6f5c64f8b6ca9ec2a88e473b21d3dd805246817e4cb30"}
Jul 25 01:20:53.282230 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.282026    2562 scope.go:117] "RemoveContainer" containerID="52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419"
Jul 25 01:20:53.283830 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.283807    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf" event={"ID":"f9afee68-ba03-440b-a6fd-067287a274f0","Type":"ContainerStarted","Data":"93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f"}
Jul 25 01:20:53.283942 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.283854    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf" event={"ID":"f9afee68-ba03-440b-a6fd-067287a274f0","Type":"ContainerStarted","Data":"ac8cd3f906577f6fe643ee11b43dab1d17ffc27855a9391c23633eb855d00add"}
Jul 25 01:20:53.283942 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.283901    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:20:53.291725 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.291707    2562 scope.go:117] "RemoveContainer" containerID="52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419"
Jul 25 01:20:53.292095 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:20:53.291962    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419\": container with ID starting with 52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419 not found: ID does not exist" containerID="52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419"
Jul 25 01:20:53.292095 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.292020    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419"} err="failed to get container status \"52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419\": rpc error: code = NotFound desc = could not find container \"52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419\": container with ID starting with 52d4ada8b2b32b07401be314319941fff7cfc19917099c19d98208a564536419 not found: ID does not exist"
Jul 25 01:20:53.323904 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.323860    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf" podStartSLOduration=1.323846692 podStartE2EDuration="1.323846692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:20:53.321818885 +0000 UTC m=+531.792775223" watchObservedRunningTime="2026-07-25 01:20:53.323846692 +0000 UTC m=+531.794803030"
Jul 25 01:20:53.339803 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.339779    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"]
Jul 25 01:20:53.348547 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:53.348525    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-n7drc"]
Jul 25 01:20:54.080385 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:54.075924    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10b45418-63b4-4707-b269-66c39aa5f6ae" path="/var/lib/kubelet/pods/10b45418-63b4-4707-b269-66c39aa5f6ae/volumes"
Jul 25 01:20:54.164296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:54.164266    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:20:54.271055 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:20:54.271003    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:04.292300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.292275    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:21:04.344164 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.344133    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"]
Jul 25 01:21:04.344340 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.344318    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb" podUID="e1d4f597-85d2-4f34-ade8-8fe40589a06f" containerName="manager" containerID="cri-o://43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab" gracePeriod=10
Jul 25 01:21:04.596461 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.596436    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:21:04.645048 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.645023    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s9vbq\" (UniqueName: \"kubernetes.io/projected/e1d4f597-85d2-4f34-ade8-8fe40589a06f-kube-api-access-s9vbq\") pod \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\" (UID: \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\") "
Jul 25 01:21:04.645170 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.645091    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1d4f597-85d2-4f34-ade8-8fe40589a06f-extensions-socket-volume\") pod \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\" (UID: \"e1d4f597-85d2-4f34-ade8-8fe40589a06f\") "
Jul 25 01:21:04.645448 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.645428    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1d4f597-85d2-4f34-ade8-8fe40589a06f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e1d4f597-85d2-4f34-ade8-8fe40589a06f" (UID: "e1d4f597-85d2-4f34-ade8-8fe40589a06f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:21:04.647032 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.646995    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1d4f597-85d2-4f34-ade8-8fe40589a06f-kube-api-access-s9vbq" (OuterVolumeSpecName: "kube-api-access-s9vbq") pod "e1d4f597-85d2-4f34-ade8-8fe40589a06f" (UID: "e1d4f597-85d2-4f34-ade8-8fe40589a06f"). InnerVolumeSpecName "kube-api-access-s9vbq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:21:04.746478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.746437    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e1d4f597-85d2-4f34-ade8-8fe40589a06f-extensions-socket-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:21:04.746478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:04.746474    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s9vbq\" (UniqueName: \"kubernetes.io/projected/e1d4f597-85d2-4f34-ade8-8fe40589a06f-kube-api-access-s9vbq\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:21:05.337769 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.337735    2562 generic.go:356] "Generic (PLEG): container finished" podID="e1d4f597-85d2-4f34-ade8-8fe40589a06f" containerID="43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab" exitCode=0
Jul 25 01:21:05.338214 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.337777    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"
Jul 25 01:21:05.338214 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.337806    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb" event={"ID":"e1d4f597-85d2-4f34-ade8-8fe40589a06f","Type":"ContainerDied","Data":"43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab"}
Jul 25 01:21:05.338214 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.337849    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb" event={"ID":"e1d4f597-85d2-4f34-ade8-8fe40589a06f","Type":"ContainerDied","Data":"9c16a76c1b26271a77f15998edbceee9b8e10bddfc84eed1a06b73fad16d6992"}
Jul 25 01:21:05.338214 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.337872    2562 scope.go:117] "RemoveContainer" containerID="43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab"
Jul 25 01:21:05.346270 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.346123    2562 scope.go:117] "RemoveContainer" containerID="43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab"
Jul 25 01:21:05.346403 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:21:05.346384    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab\": container with ID starting with 43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab not found: ID does not exist" containerID="43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab"
Jul 25 01:21:05.346445 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.346411    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab"} err="failed to get container status \"43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab\": rpc error: code = NotFound desc = could not find container \"43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab\": container with ID starting with 43142b36cecc0de9f5ee967e9d2dce0ff5db973a32b710c727a5158fce6343ab not found: ID does not exist"
Jul 25 01:21:05.359378 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.359357    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"]
Jul 25 01:21:05.363290 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.363269    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-7xkcb"]
Jul 25 01:21:05.804158 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.804126    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:05.905067 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:05.905027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:06.076000 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:06.075911    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e1d4f597-85d2-4f34-ade8-8fe40589a06f" path="/var/lib/kubelet/pods/e1d4f597-85d2-4f34-ade8-8fe40589a06f/volumes"
Jul 25 01:21:08.719436 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.719403    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"]
Jul 25 01:21:08.719816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.719731    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e1d4f597-85d2-4f34-ade8-8fe40589a06f" containerName="manager"
Jul 25 01:21:08.719816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.719741    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="e1d4f597-85d2-4f34-ade8-8fe40589a06f" containerName="manager"
Jul 25 01:21:08.719816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.719789    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="e1d4f597-85d2-4f34-ade8-8fe40589a06f" containerName="manager"
Jul 25 01:21:08.724043 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.724022    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.727003 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.726964    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-xng8r\""
Jul 25 01:21:08.735667 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.735645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"]
Jul 25 01:21:08.751853 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.751816    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:08.780659 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.780630    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.780765 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.780662    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b75df48-087c-4718-a246-d51b95fd1669-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.780765 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.780702    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pntq2\" (UniqueName: \"kubernetes.io/projected/7b75df48-087c-4718-a246-d51b95fd1669-kube-api-access-pntq2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.780765 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.780753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.781035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.780799    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.781035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.780876    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b75df48-087c-4718-a246-d51b95fd1669-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.781035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.780908    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.781035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.780934    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.781279 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.781045    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b75df48-087c-4718-a246-d51b95fd1669-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882419 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882387    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882419 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882418    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b75df48-087c-4718-a246-d51b95fd1669-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882445    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pntq2\" (UniqueName: \"kubernetes.io/projected/7b75df48-087c-4718-a246-d51b95fd1669-kube-api-access-pntq2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882465    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882488    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882530    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b75df48-087c-4718-a246-d51b95fd1669-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882548    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882566    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.882628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882593    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b75df48-087c-4718-a246-d51b95fd1669-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.883021 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882893    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.883021 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.882903    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.883232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.883210    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.883298 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.883247    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.883338 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.883308    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b75df48-087c-4718-a246-d51b95fd1669-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.884786 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.884764    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b75df48-087c-4718-a246-d51b95fd1669-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.884894 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.884877    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b75df48-087c-4718-a246-d51b95fd1669-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.891201 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.891177    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b75df48-087c-4718-a246-d51b95fd1669-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:08.891298 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:08.891244    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pntq2\" (UniqueName: \"kubernetes.io/projected/7b75df48-087c-4718-a246-d51b95fd1669-kube-api-access-pntq2\") pod \"maas-default-gateway-openshift-default-78bb78ff78-5jk8x\" (UID: \"7b75df48-087c-4718-a246-d51b95fd1669\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:09.033916 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.033835    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:09.165365 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.165341    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"]
Jul 25 01:21:09.167421 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:21:09.167394    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b75df48_087c_4718_a246_d51b95fd1669.slice/crio-97bf6ec16bb40a3b391d5e0ae585305e99fa20bfd20afdd4210f1faa9081bc15 WatchSource:0}: Error finding container 97bf6ec16bb40a3b391d5e0ae585305e99fa20bfd20afdd4210f1faa9081bc15: Status 404 returned error can't find the container with id 97bf6ec16bb40a3b391d5e0ae585305e99fa20bfd20afdd4210f1faa9081bc15
Jul 25 01:21:09.169733 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.169696    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:21:09.169810 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.169774    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:21:09.169852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.169817    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:21:09.282622 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.282583    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:09.359702 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.359627    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" event={"ID":"7b75df48-087c-4718-a246-d51b95fd1669","Type":"ContainerStarted","Data":"0fadd98491cf7ba1b3c4da721f572962eabd311a78b553ac18c48babb4eb5bda"}
Jul 25 01:21:09.359702 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.359678    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" event={"ID":"7b75df48-087c-4718-a246-d51b95fd1669","Type":"ContainerStarted","Data":"97bf6ec16bb40a3b391d5e0ae585305e99fa20bfd20afdd4210f1faa9081bc15"}
Jul 25 01:21:09.381436 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:09.381371    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" podStartSLOduration=1.381357412 podStartE2EDuration="1.381357412s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:21:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:21:09.379895486 +0000 UTC m=+547.850851841" watchObservedRunningTime="2026-07-25 01:21:09.381357412 +0000 UTC m=+547.852313749"
Jul 25 01:21:10.034011 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:10.033965    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:10.038697 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:10.038674    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:10.364417 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:10.364347    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:10.365140 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:10.365122    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:21:12.262443 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:12.262405    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:12.363800 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:12.363762    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:19.068822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:19.068786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:22.152708 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:22.152673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:22.257871 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:22.257834    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:22.361791 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:22.361752    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:23.011052 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:23.011023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:23.108454 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:23.108418    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:33.637745 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:33.637707    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:33.768007 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:33.767957    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:41.925202 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:41.925160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:42.022818 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:42.022786    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:42.393621 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:42.393577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:43.029448 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:43.029411    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:55.265827 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:55.265794    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:21:55.369739 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:21:55.369704    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:01.723814 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.723780    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:01.739285 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.739255    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-699f5ff688-kzh45"]
Jul 25 01:22:01.742599 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.742585    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:01.745457 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.745436    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 25 01:22:01.745576 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.745437    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8qvsq\""
Jul 25 01:22:01.746554 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.746537    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 25 01:22:01.752926 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.752905    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-699f5ff688-kzh45"]
Jul 25 01:22:01.805769 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.805742    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbp8x\" (UniqueName: \"kubernetes.io/projected/be0dbece-8805-4fbf-986a-72ba1b1a33a5-kube-api-access-pbp8x\") pod \"maas-controller-699f5ff688-kzh45\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") " pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:01.805924 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.805814    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert\") pod \"maas-controller-699f5ff688-kzh45\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") " pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:01.820467 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.820438    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:01.876425 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.876394    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:01.906297 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.906271    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pbp8x\" (UniqueName: \"kubernetes.io/projected/be0dbece-8805-4fbf-986a-72ba1b1a33a5-kube-api-access-pbp8x\") pod \"maas-controller-699f5ff688-kzh45\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") " pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:01.906421 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.906341    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert\") pod \"maas-controller-699f5ff688-kzh45\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") " pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:01.908892 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.908869    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 25 01:22:01.916741 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.916724    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbp8x\" (UniqueName: \"kubernetes.io/projected/be0dbece-8805-4fbf-986a-72ba1b1a33a5-kube-api-access-pbp8x\") pod \"maas-controller-699f5ff688-kzh45\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") " pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:01.916819 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:22:01.916776    2562 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found
Jul 25 01:22:01.916819 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:22:01.916814    2562 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert podName:be0dbece-8805-4fbf-986a-72ba1b1a33a5 nodeName:}" failed. No retries permitted until 2026-07-25 01:22:02.41680165 +0000 UTC m=+600.887757965 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert") pod "maas-controller-699f5ff688-kzh45" (UID: "be0dbece-8805-4fbf-986a-72ba1b1a33a5") : secret "maas-controller-webhook-cert" not found
Jul 25 01:22:01.921513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.921488    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-ch8tw"]
Jul 25 01:22:01.924933 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.924916    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:01.935108 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.935086    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-ch8tw"]
Jul 25 01:22:01.966567 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:01.966513    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:02.007733 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.007679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43d6e9f9-f1df-4f69-af81-f0477d7daf88-webhook-cert\") pod \"maas-controller-859cc5f8b9-ch8tw\" (UID: \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\") " pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:02.007733 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.007707    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xxhg6\" (UniqueName: \"kubernetes.io/projected/43d6e9f9-f1df-4f69-af81-f0477d7daf88-kube-api-access-xxhg6\") pod \"maas-controller-859cc5f8b9-ch8tw\" (UID: \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\") " pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:02.019435 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.019412    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:02.108161 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.108136    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43d6e9f9-f1df-4f69-af81-f0477d7daf88-webhook-cert\") pod \"maas-controller-859cc5f8b9-ch8tw\" (UID: \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\") " pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:02.108259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.108166    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xxhg6\" (UniqueName: \"kubernetes.io/projected/43d6e9f9-f1df-4f69-af81-f0477d7daf88-kube-api-access-xxhg6\") pod \"maas-controller-859cc5f8b9-ch8tw\" (UID: \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\") " pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:02.110231 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.110206    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43d6e9f9-f1df-4f69-af81-f0477d7daf88-webhook-cert\") pod \"maas-controller-859cc5f8b9-ch8tw\" (UID: \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\") " pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:02.115937 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.115915    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xxhg6\" (UniqueName: \"kubernetes.io/projected/43d6e9f9-f1df-4f69-af81-f0477d7daf88-kube-api-access-xxhg6\") pod \"maas-controller-859cc5f8b9-ch8tw\" (UID: \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\") " pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:02.238097 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.238079    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8qvsq\""
Jul 25 01:22:02.246739 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.246724    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:02.375030 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.375010    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-ch8tw"]
Jul 25 01:22:02.376932 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:22:02.376904    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d6e9f9_f1df_4f69_af81_f0477d7daf88.slice/crio-5cb5ba7257c04bda06a0c31d7a9cb9f1ca6252b4a78b905d7394f709c563098b WatchSource:0}: Error finding container 5cb5ba7257c04bda06a0c31d7a9cb9f1ca6252b4a78b905d7394f709c563098b: Status 404 returned error can't find the container with id 5cb5ba7257c04bda06a0c31d7a9cb9f1ca6252b4a78b905d7394f709c563098b
Jul 25 01:22:02.511738 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.511671    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert\") pod \"maas-controller-699f5ff688-kzh45\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") " pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:02.513927 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.513910    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert\") pod \"maas-controller-699f5ff688-kzh45\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") " pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:02.595814 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.595789    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw" event={"ID":"43d6e9f9-f1df-4f69-af81-f0477d7daf88","Type":"ContainerStarted","Data":"5cb5ba7257c04bda06a0c31d7a9cb9f1ca6252b4a78b905d7394f709c563098b"}
Jul 25 01:22:02.653444 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.653419    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:02.777711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:02.777685    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-699f5ff688-kzh45"]
Jul 25 01:22:02.779473 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:22:02.779447    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe0dbece_8805_4fbf_986a_72ba1b1a33a5.slice/crio-3c557461c066c037b01035f6bec7d3af028248ec4de2efc7a111a5549594d550 WatchSource:0}: Error finding container 3c557461c066c037b01035f6bec7d3af028248ec4de2efc7a111a5549594d550: Status 404 returned error can't find the container with id 3c557461c066c037b01035f6bec7d3af028248ec4de2efc7a111a5549594d550
Jul 25 01:22:03.605188 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:03.605149    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699f5ff688-kzh45" event={"ID":"be0dbece-8805-4fbf-986a-72ba1b1a33a5","Type":"ContainerStarted","Data":"3c557461c066c037b01035f6bec7d3af028248ec4de2efc7a111a5549594d550"}
Jul 25 01:22:04.823240 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:04.823214    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 25 01:22:04.988601 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:04.988541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:05.086844 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:05.086770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:05.617067 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:05.617030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw" event={"ID":"43d6e9f9-f1df-4f69-af81-f0477d7daf88","Type":"ContainerStarted","Data":"e97f2ece42e3b9353ad2f21f83cb968f0759111004f76095eca0a4e79f40c88c"}
Jul 25 01:22:05.617242 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:05.617087    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:05.618497 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:05.618476    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699f5ff688-kzh45" event={"ID":"be0dbece-8805-4fbf-986a-72ba1b1a33a5","Type":"ContainerStarted","Data":"1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22"}
Jul 25 01:22:05.618564 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:05.618525    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:05.639346 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:05.639306    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw" podStartSLOduration=2.197002989 podStartE2EDuration="4.639292138s" totalImagesPullingTime="2.442289149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:22:05.636971572 +0000 UTC m=+604.107927907" watchObservedRunningTime="2026-07-25 01:22:05.639292138 +0000 UTC m=+604.110248475"
Jul 25 01:22:05.653580 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:05.653539    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-699f5ff688-kzh45" podStartSLOduration=2.607963724 podStartE2EDuration="4.653526377s" totalImagesPullingTime="2.045562653s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:22:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:22:05.652871412 +0000 UTC m=+604.123827761" watchObservedRunningTime="2026-07-25 01:22:05.653526377 +0000 UTC m=+604.124482714"
Jul 25 01:22:16.626889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:16.626862    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:16.629215 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:16.626921    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:22:16.688027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:16.687996    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-699f5ff688-kzh45"]
Jul 25 01:22:16.688451 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:16.688409    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-699f5ff688-kzh45" podUID="be0dbece-8805-4fbf-986a-72ba1b1a33a5" containerName="manager" containerID="cri-o://1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22" gracePeriod=30
Jul 25 01:22:16.951700 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:16.951680    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:17.036865 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.036837    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert\") pod \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") "
Jul 25 01:22:17.037042 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.036903    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pbp8x\" (UniqueName: \"kubernetes.io/projected/be0dbece-8805-4fbf-986a-72ba1b1a33a5-kube-api-access-pbp8x\") pod \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\" (UID: \"be0dbece-8805-4fbf-986a-72ba1b1a33a5\") "
Jul 25 01:22:17.039113 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.039076    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "be0dbece-8805-4fbf-986a-72ba1b1a33a5" (UID: "be0dbece-8805-4fbf-986a-72ba1b1a33a5"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:22:17.039221 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.039168    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be0dbece-8805-4fbf-986a-72ba1b1a33a5-kube-api-access-pbp8x" (OuterVolumeSpecName: "kube-api-access-pbp8x") pod "be0dbece-8805-4fbf-986a-72ba1b1a33a5" (UID: "be0dbece-8805-4fbf-986a-72ba1b1a33a5"). InnerVolumeSpecName "kube-api-access-pbp8x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:22:17.138281 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.138248    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/be0dbece-8805-4fbf-986a-72ba1b1a33a5-webhook-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:22:17.138281 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.138274    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pbp8x\" (UniqueName: \"kubernetes.io/projected/be0dbece-8805-4fbf-986a-72ba1b1a33a5-kube-api-access-pbp8x\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:22:17.683127 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.683092    2562 generic.go:356] "Generic (PLEG): container finished" podID="be0dbece-8805-4fbf-986a-72ba1b1a33a5" containerID="1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22" exitCode=0
Jul 25 01:22:17.683543 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.683139    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-699f5ff688-kzh45"
Jul 25 01:22:17.683543 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.683186    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699f5ff688-kzh45" event={"ID":"be0dbece-8805-4fbf-986a-72ba1b1a33a5","Type":"ContainerDied","Data":"1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22"}
Jul 25 01:22:17.683543 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.683247    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-699f5ff688-kzh45" event={"ID":"be0dbece-8805-4fbf-986a-72ba1b1a33a5","Type":"ContainerDied","Data":"3c557461c066c037b01035f6bec7d3af028248ec4de2efc7a111a5549594d550"}
Jul 25 01:22:17.683543 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.683282    2562 scope.go:117] "RemoveContainer" containerID="1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22"
Jul 25 01:22:17.692064 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.691857    2562 scope.go:117] "RemoveContainer" containerID="1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22"
Jul 25 01:22:17.692163 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:22:17.692143    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22\": container with ID starting with 1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22 not found: ID does not exist" containerID="1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22"
Jul 25 01:22:17.692204 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.692171    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22"} err="failed to get container status \"1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22\": rpc error: code = NotFound desc = could not find container \"1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22\": container with ID starting with 1f51c8610c3ceec67c5a395ab22d718e879efcabdf23ab1fb0e638fb26345a22 not found: ID does not exist"
Jul 25 01:22:17.703571 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.703547    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-699f5ff688-kzh45"]
Jul 25 01:22:17.707242 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:17.707222    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-699f5ff688-kzh45"]
Jul 25 01:22:18.075182 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:18.075106    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be0dbece-8805-4fbf-986a-72ba1b1a33a5" path="/var/lib/kubelet/pods/be0dbece-8805-4fbf-986a-72ba1b1a33a5/volumes"
Jul 25 01:22:18.137824 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:18.137792    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:19.208530 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:19.208499    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:37.553470 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:37.553368    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:37.645833 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:37.645795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:38.247958 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.247921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:38.594690 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.594599    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:38.842443 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.842407    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:38.932711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.932673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:38.936564 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.936541    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"]
Jul 25 01:22:38.936874 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.936863    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be0dbece-8805-4fbf-986a-72ba1b1a33a5" containerName="manager"
Jul 25 01:22:38.936918 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.936875    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="be0dbece-8805-4fbf-986a-72ba1b1a33a5" containerName="manager"
Jul 25 01:22:38.936957 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.936938    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="be0dbece-8805-4fbf-986a-72ba1b1a33a5" containerName="manager"
Jul 25 01:22:38.944943 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.944922    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:38.948328 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.948307    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-q6d7p\""
Jul 25 01:22:38.959734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.959704    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Jul 25 01:22:38.960368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.960342    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"]
Jul 25 01:22:38.973594 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.973564    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-j8tq2"]
Jul 25 01:22:38.977926 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.977900    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:38.989000 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:38.988951    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-j8tq2"]
Jul 25 01:22:39.024087 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.024054    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wbvx\" (UniqueName: \"kubernetes.io/projected/48ed41f2-58a1-4f84-9c9c-82525a777da2-kube-api-access-4wbvx\") pod \"payload-processing-5cc6cd444b-j8tq2\" (UID: \"48ed41f2-58a1-4f84-9c9c-82525a777da2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:39.024253 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.024125    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec36db6-f077-4467-ba86-86b131ad2f4d-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vw2r7\" (UID: \"bec36db6-f077-4467-ba86-86b131ad2f4d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:39.024253 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.024155    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c9h4\" (UniqueName: \"kubernetes.io/projected/bec36db6-f077-4467-ba86-86b131ad2f4d-kube-api-access-4c9h4\") pod \"payload-pre-processing-687dbb6cf6-vw2r7\" (UID: \"bec36db6-f077-4467-ba86-86b131ad2f4d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:39.024253 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.024174    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48ed41f2-58a1-4f84-9c9c-82525a777da2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-j8tq2\" (UID: \"48ed41f2-58a1-4f84-9c9c-82525a777da2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:39.125540 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.125506    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wbvx\" (UniqueName: \"kubernetes.io/projected/48ed41f2-58a1-4f84-9c9c-82525a777da2-kube-api-access-4wbvx\") pod \"payload-processing-5cc6cd444b-j8tq2\" (UID: \"48ed41f2-58a1-4f84-9c9c-82525a777da2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:39.125718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.125552    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec36db6-f077-4467-ba86-86b131ad2f4d-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vw2r7\" (UID: \"bec36db6-f077-4467-ba86-86b131ad2f4d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:39.125718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.125574    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c9h4\" (UniqueName: \"kubernetes.io/projected/bec36db6-f077-4467-ba86-86b131ad2f4d-kube-api-access-4c9h4\") pod \"payload-pre-processing-687dbb6cf6-vw2r7\" (UID: \"bec36db6-f077-4467-ba86-86b131ad2f4d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:39.125718 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.125603    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48ed41f2-58a1-4f84-9c9c-82525a777da2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-j8tq2\" (UID: \"48ed41f2-58a1-4f84-9c9c-82525a777da2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:39.126515 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.126492    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/48ed41f2-58a1-4f84-9c9c-82525a777da2-plugins-config-volume\") pod \"payload-processing-5cc6cd444b-j8tq2\" (UID: \"48ed41f2-58a1-4f84-9c9c-82525a777da2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:39.126562 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.126492    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bec36db6-f077-4467-ba86-86b131ad2f4d-plugins-config-volume\") pod \"payload-pre-processing-687dbb6cf6-vw2r7\" (UID: \"bec36db6-f077-4467-ba86-86b131ad2f4d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:39.133780 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.133751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wbvx\" (UniqueName: \"kubernetes.io/projected/48ed41f2-58a1-4f84-9c9c-82525a777da2-kube-api-access-4wbvx\") pod \"payload-processing-5cc6cd444b-j8tq2\" (UID: \"48ed41f2-58a1-4f84-9c9c-82525a777da2\") " pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:39.133889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.133751    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c9h4\" (UniqueName: \"kubernetes.io/projected/bec36db6-f077-4467-ba86-86b131ad2f4d-kube-api-access-4c9h4\") pod \"payload-pre-processing-687dbb6cf6-vw2r7\" (UID: \"bec36db6-f077-4467-ba86-86b131ad2f4d\") " pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:39.263161 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.263080    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:39.288960 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.288932    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:39.427377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.427117    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"]
Jul 25 01:22:39.433490 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:22:39.433451    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbec36db6_f077_4467_ba86_86b131ad2f4d.slice/crio-8d66a8f1b87303517d0cd2df1c37d4b22ec8bbea7f8b11b2da0f660d59f609ea WatchSource:0}: Error finding container 8d66a8f1b87303517d0cd2df1c37d4b22ec8bbea7f8b11b2da0f660d59f609ea: Status 404 returned error can't find the container with id 8d66a8f1b87303517d0cd2df1c37d4b22ec8bbea7f8b11b2da0f660d59f609ea
Jul 25 01:22:39.454209 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.453395    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5cc6cd444b-j8tq2"]
Jul 25 01:22:39.456238 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:22:39.456199    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ed41f2_58a1_4f84_9c9c_82525a777da2.slice/crio-8e6c009120c55c4b9a529aa5652440775492cb3b3906219f25a58d15ac4596f2 WatchSource:0}: Error finding container 8e6c009120c55c4b9a529aa5652440775492cb3b3906219f25a58d15ac4596f2: Status 404 returned error can't find the container with id 8e6c009120c55c4b9a529aa5652440775492cb3b3906219f25a58d15ac4596f2
Jul 25 01:22:39.783487 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.783448    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7" event={"ID":"bec36db6-f077-4467-ba86-86b131ad2f4d","Type":"ContainerStarted","Data":"8d66a8f1b87303517d0cd2df1c37d4b22ec8bbea7f8b11b2da0f660d59f609ea"}
Jul 25 01:22:39.784693 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:39.784672    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2" event={"ID":"48ed41f2-58a1-4f84-9c9c-82525a777da2","Type":"ContainerStarted","Data":"8e6c009120c55c4b9a529aa5652440775492cb3b3906219f25a58d15ac4596f2"}
Jul 25 01:22:42.805649 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:42.805616    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7" event={"ID":"bec36db6-f077-4467-ba86-86b131ad2f4d","Type":"ContainerStarted","Data":"4c4482a52b6ad1e7787580a5785cb73d7ecf9e8df27566aec3aee154897cd1d6"}
Jul 25 01:22:42.806121 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:42.805744    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:42.807050 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:42.807030    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2" event={"ID":"48ed41f2-58a1-4f84-9c9c-82525a777da2","Type":"ContainerStarted","Data":"2daeb427ee56c52a3c9b3baed4214460960896d96dab037a7fc593c4501d0d47"}
Jul 25 01:22:42.807194 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:42.807180    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:42.821061 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:42.821023    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7" podStartSLOduration=1.653460768 podStartE2EDuration="4.821009381s" totalImagesPullingTime="3.167548613s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:22:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:22:42.820328419 +0000 UTC m=+641.291284757" watchObservedRunningTime="2026-07-25 01:22:42.821009381 +0000 UTC m=+641.291965718"
Jul 25 01:22:42.835587 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:42.835544    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2" podStartSLOduration=1.698899395 podStartE2EDuration="4.835531456s" totalImagesPullingTime="3.136632061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:22:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:22:42.834378302 +0000 UTC m=+641.305334639" watchObservedRunningTime="2026-07-25 01:22:42.835531456 +0000 UTC m=+641.306487794"
Jul 25 01:22:46.828307 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:46.828213    2562 generic.go:356] "Generic (PLEG): container finished" podID="7b75df48-087c-4718-a246-d51b95fd1669" containerID="0fadd98491cf7ba1b3c4da721f572962eabd311a78b553ac18c48babb4eb5bda" exitCode=0
Jul 25 01:22:46.828307 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:46.828292    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" event={"ID":"7b75df48-087c-4718-a246-d51b95fd1669","Type":"ContainerDied","Data":"0fadd98491cf7ba1b3c4da721f572962eabd311a78b553ac18c48babb4eb5bda"}
Jul 25 01:22:46.828787 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:46.828675    2562 scope.go:117] "RemoveContainer" containerID="0fadd98491cf7ba1b3c4da721f572962eabd311a78b553ac18c48babb4eb5bda"
Jul 25 01:22:46.829573 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:46.829551    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:22:46.829668 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:46.829618    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:22:46.829668 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:46.829650    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:22:47.834202 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:47.834167    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" event={"ID":"7b75df48-087c-4718-a246-d51b95fd1669","Type":"ContainerStarted","Data":"8d6564ec25b299ea5282a52b3bb0f63935c41848ede47e00f918517233fa1d1e"}
Jul 25 01:22:48.034605 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:48.034572    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:22:48.034835 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:48.034817    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5jk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 25 01:22:48.034890 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:48.034861    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" podUID="7b75df48-087c-4718-a246-d51b95fd1669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 25 01:22:49.034881 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:49.034847    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5jk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 25 01:22:49.035273 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:49.034899    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" podUID="7b75df48-087c-4718-a246-d51b95fd1669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 25 01:22:49.174786 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:49.174743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:49.273948 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:49.273910    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:50.034799 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:50.034768    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5jk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 25 01:22:50.034961 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:50.034822    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" podUID="7b75df48-087c-4718-a246-d51b95fd1669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 25 01:22:51.034177 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:51.034145    2562 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-5jk8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused" start-of-body=
Jul 25 01:22:51.034337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:51.034197    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x" podUID="7b75df48-087c-4718-a246-d51b95fd1669" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.34:15021/healthz/ready\": dial tcp 10.134.0.34:15021: connect: connection refused"
Jul 25 01:22:52.038770 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:52.038740    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:22:52.039216 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:52.039008    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:22:52.039666 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:52.039651    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-5jk8x"
Jul 25 01:22:53.815368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:53.815330    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5cc6cd444b-j8tq2"
Jul 25 01:22:53.815790 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:53.815608    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-687dbb6cf6-vw2r7"
Jul 25 01:22:54.083705 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.083614    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"]
Jul 25 01:22:54.089076 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.089046    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.093322 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.093296    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Jul 25 01:22:54.093696 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.093679    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Jul 25 01:22:54.094829 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.094800    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Jul 25 01:22:54.094992 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.094883    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-ls94p\""
Jul 25 01:22:54.095424 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.095401    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"]
Jul 25 01:22:54.112684 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.112657    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:22:54.191861 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.191828    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.191861 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.191861    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ea409a-261f-40af-a816-e179423df4b2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.192092 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.191881    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.192092 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.191936    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.192092 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.192043    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcrc5\" (UniqueName: \"kubernetes.io/projected/d8ea409a-261f-40af-a816-e179423df4b2-kube-api-access-tcrc5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.192092 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.192090    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.293554 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.293526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.293734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.293592    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.293734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.293626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ea409a-261f-40af-a816-e179423df4b2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.293734 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.293651    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.293894 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.293681    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.293894 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.293803    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcrc5\" (UniqueName: \"kubernetes.io/projected/d8ea409a-261f-40af-a816-e179423df4b2-kube-api-access-tcrc5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.294069 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.293957    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.294069 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.294044    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.294225 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.294207    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.295945 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.295926    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d8ea409a-261f-40af-a816-e179423df4b2-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.296215 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.296197    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d8ea409a-261f-40af-a816-e179423df4b2-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.301500 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.301473    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcrc5\" (UniqueName: \"kubernetes.io/projected/d8ea409a-261f-40af-a816-e179423df4b2-kube-api-access-tcrc5\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv\" (UID: \"d8ea409a-261f-40af-a816-e179423df4b2\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.402174 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.402151    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:22:54.533005 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.532967    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"]
Jul 25 01:22:54.534935 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:22:54.534892    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8ea409a_261f_40af_a816_e179423df4b2.slice/crio-b06e2fbfebc36ecadb0d35a90769bc7bfdcc65552e6ab0ed9c8626477a32c358 WatchSource:0}: Error finding container b06e2fbfebc36ecadb0d35a90769bc7bfdcc65552e6ab0ed9c8626477a32c358: Status 404 returned error can't find the container with id b06e2fbfebc36ecadb0d35a90769bc7bfdcc65552e6ab0ed9c8626477a32c358
Jul 25 01:22:54.874691 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:54.874655    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv" event={"ID":"d8ea409a-261f-40af-a816-e179423df4b2","Type":"ContainerStarted","Data":"b06e2fbfebc36ecadb0d35a90769bc7bfdcc65552e6ab0ed9c8626477a32c358"}
Jul 25 01:22:55.881154 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:55.881118    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv" event={"ID":"d8ea409a-261f-40af-a816-e179423df4b2","Type":"ContainerStarted","Data":"c61dc63f41e79be6cab85b7562c68306010d0200f30d17d318c3a6e6f4d9caa5"}
Jul 25 01:22:55.899118 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:22:55.899072    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv" podStartSLOduration=0.927921101 podStartE2EDuration="1.899054641s" totalImagesPullingTime="971.13354ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:22:55.897631371 +0000 UTC m=+654.368587706" watchObservedRunningTime="2026-07-25 01:22:55.899054641 +0000 UTC m=+654.370010978"
Jul 25 01:23:04.403197 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:04.403168    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:23:04.403582 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:04.403216    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:23:04.415742 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:04.415718    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:23:04.931870 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:04.931844    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv"
Jul 25 01:23:06.312474 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:06.312438    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:23:13.788733 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:13.788701    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:23:20.692684 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.692639    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"]
Jul 25 01:23:20.699916 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.699898    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.702558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.702530    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\""
Jul 25 01:23:20.712590 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.712568    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"]
Jul 25 01:23:20.784799 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.784751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:23:20.804883 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.804861    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.805046 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.804895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rv78c\" (UniqueName: \"kubernetes.io/projected/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-kube-api-access-rv78c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.805046 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.804931    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.805046 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.805005    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.805189 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.805054    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.805189 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.805082    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.906492 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.906461    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.906652 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.906501    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rv78c\" (UniqueName: \"kubernetes.io/projected/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-kube-api-access-rv78c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.906652 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.906632    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.906735 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.906677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.906784 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.906729    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.906784 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.906766    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.906919 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.906902    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.907032 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.906967    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.907099 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.907074    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.908814 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.908791    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.909278 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.909257    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:20.915013 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:20.914969    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rv78c\" (UniqueName: \"kubernetes.io/projected/9cca9bcf-c1dc-4dd1-927b-762d26f7898f-kube-api-access-rv78c\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q\" (UID: \"9cca9bcf-c1dc-4dd1-927b-762d26f7898f\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:21.010104 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:21.010038    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:21.136835 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:21.136812    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"]
Jul 25 01:23:21.139371 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:23:21.139340    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cca9bcf_c1dc_4dd1_927b_762d26f7898f.slice/crio-ec35f077cf7af70022301388b14fe7ece20c60c62af2f26f26b360251b49ca15 WatchSource:0}: Error finding container ec35f077cf7af70022301388b14fe7ece20c60c62af2f26f26b360251b49ca15: Status 404 returned error can't find the container with id ec35f077cf7af70022301388b14fe7ece20c60c62af2f26f26b360251b49ca15
Jul 25 01:23:21.141042 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:21.141024    2562 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Jul 25 01:23:22.002667 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:22.002631    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q" event={"ID":"9cca9bcf-c1dc-4dd1-927b-762d26f7898f","Type":"ContainerStarted","Data":"0563d58abbd7b6ec72f6beb07232825aafcfcfe14a58097629af167e89fd42cb"}
Jul 25 01:23:22.003138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:22.002717    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q" event={"ID":"9cca9bcf-c1dc-4dd1-927b-762d26f7898f","Type":"ContainerStarted","Data":"ec35f077cf7af70022301388b14fe7ece20c60c62af2f26f26b360251b49ca15"}
Jul 25 01:23:22.021635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:22.021584    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q" podStartSLOduration=1.85419661 podStartE2EDuration="2.021570575s" totalImagesPullingTime="167.373965ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:23:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:23:22.018910979 +0000 UTC m=+680.489867311" watchObservedRunningTime="2026-07-25 01:23:22.021570575 +0000 UTC m=+680.492526912"
Jul 25 01:23:30.977850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:30.977819    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"]
Jul 25 01:23:30.982594 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:30.982575    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:30.985233 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:30.985205    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Jul 25 01:23:30.993386 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:30.993363    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"]
Jul 25 01:23:31.010348 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.010328    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:31.010470 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.010372    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:31.014341 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.014297    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:23:31.024127 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.024109    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:31.056055 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.056031    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q"
Jul 25 01:23:31.090627 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.090601    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.090745 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.090657    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.090792 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.090754    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eda39df-2fc7-4126-92ef-851c6431cf38-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.090837 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.090821    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddgdv\" (UniqueName: \"kubernetes.io/projected/7eda39df-2fc7-4126-92ef-851c6431cf38-kube-api-access-ddgdv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.090875 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.090853    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.091026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.090972    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192400 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192367    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eda39df-2fc7-4126-92ef-851c6431cf38-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192490    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddgdv\" (UniqueName: \"kubernetes.io/projected/7eda39df-2fc7-4126-92ef-851c6431cf38-kube-api-access-ddgdv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192526    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192565    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192793 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192621    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192852 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192821    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192916 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192895    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.192969 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.192939    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.195138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.195106    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7eda39df-2fc7-4126-92ef-851c6431cf38-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.195287 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.195155    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/7eda39df-2fc7-4126-92ef-851c6431cf38-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.200323 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.200299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddgdv\" (UniqueName: \"kubernetes.io/projected/7eda39df-2fc7-4126-92ef-851c6431cf38-kube-api-access-ddgdv\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-ds5d2\" (UID: \"7eda39df-2fc7-4126-92ef-851c6431cf38\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.296211 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.296154    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:31.429498 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:31.429473    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"]
Jul 25 01:23:31.431245 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:23:31.431218    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eda39df_2fc7_4126_92ef_851c6431cf38.slice/crio-18141c53a994bfe74adf272651d0477bae061dbd186ae5c5657a2c0c3503350a WatchSource:0}: Error finding container 18141c53a994bfe74adf272651d0477bae061dbd186ae5c5657a2c0c3503350a: Status 404 returned error can't find the container with id 18141c53a994bfe74adf272651d0477bae061dbd186ae5c5657a2c0c3503350a
Jul 25 01:23:32.051549 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:32.051514    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2" event={"ID":"7eda39df-2fc7-4126-92ef-851c6431cf38","Type":"ContainerStarted","Data":"c9060c924c8fa5dcdc1acd60efd4cc6909b556c6a8668ca62e21d00deb4f9cd3"}
Jul 25 01:23:32.051935 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:32.051562    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2" event={"ID":"7eda39df-2fc7-4126-92ef-851c6431cf38","Type":"ContainerStarted","Data":"18141c53a994bfe74adf272651d0477bae061dbd186ae5c5657a2c0c3503350a"}
Jul 25 01:23:32.068499 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:32.068440    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2" podStartSLOduration=1.887289102 podStartE2EDuration="2.068421391s" totalImagesPullingTime="181.132289ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:23:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:23:32.0682027 +0000 UTC m=+690.539159039" watchObservedRunningTime="2026-07-25 01:23:32.068421391 +0000 UTC m=+690.539377730"
Jul 25 01:23:41.296997 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:41.296948    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:41.297366 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:41.297066    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:41.309348 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:41.309323    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:42.110644 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:42.110617    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-ds5d2"
Jul 25 01:23:45.431260 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:23:45.431213    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:24:56.402372 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:24:56.402335    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:24:56.504999 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:24:56.504947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:24:57.653366 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:24:57.653338    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:24:57.746143 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:24:57.746108    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:25:43.458300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.458267    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-ch8tw"]
Jul 25 01:25:43.459202 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.458505    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw" podUID="43d6e9f9-f1df-4f69-af81-f0477d7daf88" containerName="manager" containerID="cri-o://e97f2ece42e3b9353ad2f21f83cb968f0759111004f76095eca0a4e79f40c88c" gracePeriod=30
Jul 25 01:25:43.652491 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.652457    2562 generic.go:356] "Generic (PLEG): container finished" podID="43d6e9f9-f1df-4f69-af81-f0477d7daf88" containerID="e97f2ece42e3b9353ad2f21f83cb968f0759111004f76095eca0a4e79f40c88c" exitCode=0
Jul 25 01:25:43.652645 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.652512    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw" event={"ID":"43d6e9f9-f1df-4f69-af81-f0477d7daf88","Type":"ContainerDied","Data":"e97f2ece42e3b9353ad2f21f83cb968f0759111004f76095eca0a4e79f40c88c"}
Jul 25 01:25:43.708609 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.708556    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:25:43.800015 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.799970    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xxhg6\" (UniqueName: \"kubernetes.io/projected/43d6e9f9-f1df-4f69-af81-f0477d7daf88-kube-api-access-xxhg6\") pod \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\" (UID: \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\") "
Jul 25 01:25:43.800167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.800035    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43d6e9f9-f1df-4f69-af81-f0477d7daf88-webhook-cert\") pod \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\" (UID: \"43d6e9f9-f1df-4f69-af81-f0477d7daf88\") "
Jul 25 01:25:43.802009 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.801947    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43d6e9f9-f1df-4f69-af81-f0477d7daf88-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "43d6e9f9-f1df-4f69-af81-f0477d7daf88" (UID: "43d6e9f9-f1df-4f69-af81-f0477d7daf88"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:25:43.802142 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.802019    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d6e9f9-f1df-4f69-af81-f0477d7daf88-kube-api-access-xxhg6" (OuterVolumeSpecName: "kube-api-access-xxhg6") pod "43d6e9f9-f1df-4f69-af81-f0477d7daf88" (UID: "43d6e9f9-f1df-4f69-af81-f0477d7daf88"). InnerVolumeSpecName "kube-api-access-xxhg6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:25:43.901184 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.901153    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xxhg6\" (UniqueName: \"kubernetes.io/projected/43d6e9f9-f1df-4f69-af81-f0477d7daf88-kube-api-access-xxhg6\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:25:43.901184 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:43.901183    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/43d6e9f9-f1df-4f69-af81-f0477d7daf88-webhook-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:25:44.658793 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:44.658767    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw"
Jul 25 01:25:44.659313 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:44.658822    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-ch8tw" event={"ID":"43d6e9f9-f1df-4f69-af81-f0477d7daf88","Type":"ContainerDied","Data":"5cb5ba7257c04bda06a0c31d7a9cb9f1ca6252b4a78b905d7394f709c563098b"}
Jul 25 01:25:44.659313 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:44.658899    2562 scope.go:117] "RemoveContainer" containerID="e97f2ece42e3b9353ad2f21f83cb968f0759111004f76095eca0a4e79f40c88c"
Jul 25 01:25:44.677965 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:44.677941    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-ch8tw"]
Jul 25 01:25:44.682094 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:44.682073    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-ch8tw"]
Jul 25 01:25:46.076205 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:25:46.076175    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d6e9f9-f1df-4f69-af81-f0477d7daf88" path="/var/lib/kubelet/pods/43d6e9f9-f1df-4f69-af81-f0477d7daf88/volumes"
Jul 25 01:26:37.802635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:37.802603    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:26:45.468752 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.468721    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-4hvns"]
Jul 25 01:26:45.469241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.469103    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d6e9f9-f1df-4f69-af81-f0477d7daf88" containerName="manager"
Jul 25 01:26:45.469241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.469114    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d6e9f9-f1df-4f69-af81-f0477d7daf88" containerName="manager"
Jul 25 01:26:45.469241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.469176    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d6e9f9-f1df-4f69-af81-f0477d7daf88" containerName="manager"
Jul 25 01:26:45.472152 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.472136    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:45.476232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.476204    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8qvsq\""
Jul 25 01:26:45.476382 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.476240    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 25 01:26:45.476382 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.476344    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 25 01:26:45.481252 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.481233    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-4hvns"]
Jul 25 01:26:45.548243 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.548201    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bd9df50-0f1c-469d-8400-e5bcf006263d-webhook-cert\") pod \"maas-controller-859cc5f8b9-4hvns\" (UID: \"8bd9df50-0f1c-469d-8400-e5bcf006263d\") " pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:45.548430 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.548251    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4rts\" (UniqueName: \"kubernetes.io/projected/8bd9df50-0f1c-469d-8400-e5bcf006263d-kube-api-access-b4rts\") pod \"maas-controller-859cc5f8b9-4hvns\" (UID: \"8bd9df50-0f1c-469d-8400-e5bcf006263d\") " pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:45.649255 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.649219    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bd9df50-0f1c-469d-8400-e5bcf006263d-webhook-cert\") pod \"maas-controller-859cc5f8b9-4hvns\" (UID: \"8bd9df50-0f1c-469d-8400-e5bcf006263d\") " pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:45.649255 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.649255    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4rts\" (UniqueName: \"kubernetes.io/projected/8bd9df50-0f1c-469d-8400-e5bcf006263d-kube-api-access-b4rts\") pod \"maas-controller-859cc5f8b9-4hvns\" (UID: \"8bd9df50-0f1c-469d-8400-e5bcf006263d\") " pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:45.651721 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.651701    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bd9df50-0f1c-469d-8400-e5bcf006263d-webhook-cert\") pod \"maas-controller-859cc5f8b9-4hvns\" (UID: \"8bd9df50-0f1c-469d-8400-e5bcf006263d\") " pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:45.657595 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.657576    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4rts\" (UniqueName: \"kubernetes.io/projected/8bd9df50-0f1c-469d-8400-e5bcf006263d-kube-api-access-b4rts\") pod \"maas-controller-859cc5f8b9-4hvns\" (UID: \"8bd9df50-0f1c-469d-8400-e5bcf006263d\") " pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:45.784220 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.784137    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:45.917737 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.917712    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-4hvns"]
Jul 25 01:26:45.919813 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:26:45.919791    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8bd9df50_0f1c_469d_8400_e5bcf006263d.slice/crio-0fdb98e1a84ab710f929a297813f607fe90084f3fa3a4134cce4caf0b8c75e75 WatchSource:0}: Error finding container 0fdb98e1a84ab710f929a297813f607fe90084f3fa3a4134cce4caf0b8c75e75: Status 404 returned error can't find the container with id 0fdb98e1a84ab710f929a297813f607fe90084f3fa3a4134cce4caf0b8c75e75
Jul 25 01:26:45.942584 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:45.942551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-4hvns" event={"ID":"8bd9df50-0f1c-469d-8400-e5bcf006263d","Type":"ContainerStarted","Data":"0fdb98e1a84ab710f929a297813f607fe90084f3fa3a4134cce4caf0b8c75e75"}
Jul 25 01:26:46.948389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:46.948302    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-4hvns" event={"ID":"8bd9df50-0f1c-469d-8400-e5bcf006263d","Type":"ContainerStarted","Data":"9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49"}
Jul 25 01:26:46.948389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:46.948371    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:26:46.968462 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:46.968422    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-859cc5f8b9-4hvns" podStartSLOduration=1.484140343 podStartE2EDuration="1.968404487s" totalImagesPullingTime="484.264144ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:26:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:26:46.967744588 +0000 UTC m=+885.438700925" watchObservedRunningTime="2026-07-25 01:26:46.968404487 +0000 UTC m=+885.439360826"
Jul 25 01:26:57.956214 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:26:57.956140    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:27:02.496143 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.496097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:02.604594 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.604543    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-4hvns"]
Jul 25 01:27:02.604809 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.604765    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-859cc5f8b9-4hvns" podUID="8bd9df50-0f1c-469d-8400-e5bcf006263d" containerName="manager" containerID="cri-o://9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49" gracePeriod=30
Jul 25 01:27:02.851026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.850967    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:27:02.890860 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.890826    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bd9df50-0f1c-469d-8400-e5bcf006263d-webhook-cert\") pod \"8bd9df50-0f1c-469d-8400-e5bcf006263d\" (UID: \"8bd9df50-0f1c-469d-8400-e5bcf006263d\") "
Jul 25 01:27:02.891062 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.890877    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4rts\" (UniqueName: \"kubernetes.io/projected/8bd9df50-0f1c-469d-8400-e5bcf006263d-kube-api-access-b4rts\") pod \"8bd9df50-0f1c-469d-8400-e5bcf006263d\" (UID: \"8bd9df50-0f1c-469d-8400-e5bcf006263d\") "
Jul 25 01:27:02.893169 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.893136    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8bd9df50-0f1c-469d-8400-e5bcf006263d-kube-api-access-b4rts" (OuterVolumeSpecName: "kube-api-access-b4rts") pod "8bd9df50-0f1c-469d-8400-e5bcf006263d" (UID: "8bd9df50-0f1c-469d-8400-e5bcf006263d"). InnerVolumeSpecName "kube-api-access-b4rts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:27:02.893407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.893391    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8bd9df50-0f1c-469d-8400-e5bcf006263d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "8bd9df50-0f1c-469d-8400-e5bcf006263d" (UID: "8bd9df50-0f1c-469d-8400-e5bcf006263d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Jul 25 01:27:02.991906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.991865    2562 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8bd9df50-0f1c-469d-8400-e5bcf006263d-webhook-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:27:02.991906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:02.991906    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4rts\" (UniqueName: \"kubernetes.io/projected/8bd9df50-0f1c-469d-8400-e5bcf006263d-kube-api-access-b4rts\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:27:03.030040 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.029953    2562 generic.go:356] "Generic (PLEG): container finished" podID="8bd9df50-0f1c-469d-8400-e5bcf006263d" containerID="9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49" exitCode=0
Jul 25 01:27:03.030040 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.030009    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-4hvns"
Jul 25 01:27:03.030256 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.030045    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-4hvns" event={"ID":"8bd9df50-0f1c-469d-8400-e5bcf006263d","Type":"ContainerDied","Data":"9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49"}
Jul 25 01:27:03.030256 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.030101    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-4hvns" event={"ID":"8bd9df50-0f1c-469d-8400-e5bcf006263d","Type":"ContainerDied","Data":"0fdb98e1a84ab710f929a297813f607fe90084f3fa3a4134cce4caf0b8c75e75"}
Jul 25 01:27:03.030256 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.030125    2562 scope.go:117] "RemoveContainer" containerID="9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49"
Jul 25 01:27:03.040484 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.040458    2562 scope.go:117] "RemoveContainer" containerID="9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49"
Jul 25 01:27:03.040823 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:27:03.040798    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49\": container with ID starting with 9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49 not found: ID does not exist" containerID="9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49"
Jul 25 01:27:03.040902 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.040835    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49"} err="failed to get container status \"9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49\": rpc error: code = NotFound desc = could not find container \"9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49\": container with ID starting with 9812c4a99edce70372df879abad59ab5b7170120f6715bfda29beca5b989de49 not found: ID does not exist"
Jul 25 01:27:03.053471 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.053441    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-4hvns"]
Jul 25 01:27:03.056811 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.056782    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-4hvns"]
Jul 25 01:27:03.735483 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.735452    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:03.856485 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.856454    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-n48bk"]
Jul 25 01:27:03.856856 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.856838    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8bd9df50-0f1c-469d-8400-e5bcf006263d" containerName="manager"
Jul 25 01:27:03.856856 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.856855    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8bd9df50-0f1c-469d-8400-e5bcf006263d" containerName="manager"
Jul 25 01:27:03.857178 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.856932    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8bd9df50-0f1c-469d-8400-e5bcf006263d" containerName="manager"
Jul 25 01:27:03.861293 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.861273    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:03.865220 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.865203    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-8qvsq\""
Jul 25 01:27:03.865220 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.865218    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Jul 25 01:27:03.865366 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.865205    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Jul 25 01:27:03.872290 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.872056    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-n48bk"]
Jul 25 01:27:03.899776 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.899746    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d05574de-837a-42b0-84da-65f4210f7d12-webhook-cert\") pod \"maas-controller-859cc5f8b9-n48bk\" (UID: \"d05574de-837a-42b0-84da-65f4210f7d12\") " pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:03.899934 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:03.899826    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bsjxb\" (UniqueName: \"kubernetes.io/projected/d05574de-837a-42b0-84da-65f4210f7d12-kube-api-access-bsjxb\") pod \"maas-controller-859cc5f8b9-n48bk\" (UID: \"d05574de-837a-42b0-84da-65f4210f7d12\") " pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:04.000509 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:04.000426    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bsjxb\" (UniqueName: \"kubernetes.io/projected/d05574de-837a-42b0-84da-65f4210f7d12-kube-api-access-bsjxb\") pod \"maas-controller-859cc5f8b9-n48bk\" (UID: \"d05574de-837a-42b0-84da-65f4210f7d12\") " pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:04.000509 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:04.000487    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d05574de-837a-42b0-84da-65f4210f7d12-webhook-cert\") pod \"maas-controller-859cc5f8b9-n48bk\" (UID: \"d05574de-837a-42b0-84da-65f4210f7d12\") " pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:04.002846 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:04.002819    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d05574de-837a-42b0-84da-65f4210f7d12-webhook-cert\") pod \"maas-controller-859cc5f8b9-n48bk\" (UID: \"d05574de-837a-42b0-84da-65f4210f7d12\") " pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:04.008058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:04.008037    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bsjxb\" (UniqueName: \"kubernetes.io/projected/d05574de-837a-42b0-84da-65f4210f7d12-kube-api-access-bsjxb\") pod \"maas-controller-859cc5f8b9-n48bk\" (UID: \"d05574de-837a-42b0-84da-65f4210f7d12\") " pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:04.075516 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:04.075483    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8bd9df50-0f1c-469d-8400-e5bcf006263d" path="/var/lib/kubelet/pods/8bd9df50-0f1c-469d-8400-e5bcf006263d/volumes"
Jul 25 01:27:04.180256 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:04.180227    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:04.309916 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:04.309879    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-859cc5f8b9-n48bk"]
Jul 25 01:27:04.312534 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:27:04.312507    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd05574de_837a_42b0_84da_65f4210f7d12.slice/crio-3dbff8a932c6db1709201d3732d59d53b9e8ce4fcc79ed214a89a71318a9f30b WatchSource:0}: Error finding container 3dbff8a932c6db1709201d3732d59d53b9e8ce4fcc79ed214a89a71318a9f30b: Status 404 returned error can't find the container with id 3dbff8a932c6db1709201d3732d59d53b9e8ce4fcc79ed214a89a71318a9f30b
Jul 25 01:27:05.043360 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:05.043325    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-n48bk" event={"ID":"d05574de-837a-42b0-84da-65f4210f7d12","Type":"ContainerStarted","Data":"ea881082577f3d2f802f772107881e2bd4e39c167811437672db8f6208c6bde5"}
Jul 25 01:27:05.043801 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:05.043379    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-859cc5f8b9-n48bk" event={"ID":"d05574de-837a-42b0-84da-65f4210f7d12","Type":"ContainerStarted","Data":"3dbff8a932c6db1709201d3732d59d53b9e8ce4fcc79ed214a89a71318a9f30b"}
Jul 25 01:27:05.043801 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:05.043406    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:05.076344 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:05.076242    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-859cc5f8b9-n48bk" podStartSLOduration=1.571914559 podStartE2EDuration="2.076227153s" totalImagesPullingTime="504.312594ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:27:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:27:05.074416524 +0000 UTC m=+903.545372863" watchObservedRunningTime="2026-07-25 01:27:05.076227153 +0000 UTC m=+903.547183490"
Jul 25 01:27:16.051118 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:16.051082    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-859cc5f8b9-n48bk"
Jul 25 01:27:45.108216 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:45.108183    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:46.288667 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:46.288635    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:51.336360 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:51.336325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:51.399730 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:51.399702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:51.477889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:51.477851    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:51.575440 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:51.575397    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:51.712457 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:51.712409    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:57.583214 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:57.583153    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:27:57.642331 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:27:57.642295    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:46.342622 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:30:46.342573    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:46.398206 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:30:46.398178    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:30:58.615888 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:30:58.615843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:32:10.056380 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:10.056342    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"]
Jul 25 01:32:10.056903 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:10.056619    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf" podUID="f9afee68-ba03-440b-a6fd-067287a274f0" containerName="manager" containerID="cri-o://93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f" gracePeriod=10
Jul 25 01:32:12.310442 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.310418    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:32:12.418790 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.418751    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9afee68-ba03-440b-a6fd-067287a274f0-extensions-socket-volume\") pod \"f9afee68-ba03-440b-a6fd-067287a274f0\" (UID: \"f9afee68-ba03-440b-a6fd-067287a274f0\") "
Jul 25 01:32:12.418790 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.418796    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ldgqx\" (UniqueName: \"kubernetes.io/projected/f9afee68-ba03-440b-a6fd-067287a274f0-kube-api-access-ldgqx\") pod \"f9afee68-ba03-440b-a6fd-067287a274f0\" (UID: \"f9afee68-ba03-440b-a6fd-067287a274f0\") "
Jul 25 01:32:12.419173 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.419147    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9afee68-ba03-440b-a6fd-067287a274f0-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f9afee68-ba03-440b-a6fd-067287a274f0" (UID: "f9afee68-ba03-440b-a6fd-067287a274f0"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:32:12.420967 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.420916    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9afee68-ba03-440b-a6fd-067287a274f0-kube-api-access-ldgqx" (OuterVolumeSpecName: "kube-api-access-ldgqx") pod "f9afee68-ba03-440b-a6fd-067287a274f0" (UID: "f9afee68-ba03-440b-a6fd-067287a274f0"). InnerVolumeSpecName "kube-api-access-ldgqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:32:12.484283 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.484251    2562 generic.go:356] "Generic (PLEG): container finished" podID="f9afee68-ba03-440b-a6fd-067287a274f0" containerID="93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f" exitCode=0
Jul 25 01:32:12.484481 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.484298    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"
Jul 25 01:32:12.484481 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.484323    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf" event={"ID":"f9afee68-ba03-440b-a6fd-067287a274f0","Type":"ContainerDied","Data":"93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f"}
Jul 25 01:32:12.484481 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.484366    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf" event={"ID":"f9afee68-ba03-440b-a6fd-067287a274f0","Type":"ContainerDied","Data":"ac8cd3f906577f6fe643ee11b43dab1d17ffc27855a9391c23633eb855d00add"}
Jul 25 01:32:12.484481 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.484388    2562 scope.go:117] "RemoveContainer" containerID="93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f"
Jul 25 01:32:12.494224 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.494207    2562 scope.go:117] "RemoveContainer" containerID="93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f"
Jul 25 01:32:12.494458 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:32:12.494442    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f\": container with ID starting with 93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f not found: ID does not exist" containerID="93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f"
Jul 25 01:32:12.494529 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.494466    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f"} err="failed to get container status \"93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f\": rpc error: code = NotFound desc = could not find container \"93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f\": container with ID starting with 93441fc10838439bd975bb40031d8a3916c0982548a1080247b2c4da1179b85f not found: ID does not exist"
Jul 25 01:32:12.504945 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.504915    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"]
Jul 25 01:32:12.509261 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.509237    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qhvjf"]
Jul 25 01:32:12.519810 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.519789    2562 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9afee68-ba03-440b-a6fd-067287a274f0-extensions-socket-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:32:12.519899 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:12.519809    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldgqx\" (UniqueName: \"kubernetes.io/projected/f9afee68-ba03-440b-a6fd-067287a274f0-kube-api-access-ldgqx\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:32:14.076092 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:14.076062    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9afee68-ba03-440b-a6fd-067287a274f0" path="/var/lib/kubelet/pods/f9afee68-ba03-440b-a6fd-067287a274f0/volumes"
Jul 25 01:32:59.755740 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:32:59.755664    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:12.724159 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.724129    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"]
Jul 25 01:34:12.724622 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.724518    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9afee68-ba03-440b-a6fd-067287a274f0" containerName="manager"
Jul 25 01:34:12.724622 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.724531    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9afee68-ba03-440b-a6fd-067287a274f0" containerName="manager"
Jul 25 01:34:12.724622 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.724603    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9afee68-ba03-440b-a6fd-067287a274f0" containerName="manager"
Jul 25 01:34:12.727653 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.727635    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:12.729995 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.729959    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-sjbxw\""
Jul 25 01:34:12.748920 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.748883    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"]
Jul 25 01:34:12.863373 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.863336    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blg74\" (UniqueName: \"kubernetes.io/projected/bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb-kube-api-access-blg74\") pod \"kuadrant-operator-controller-manager-86878c769f-scbkv\" (UID: \"bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:12.863540 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.863489    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-scbkv\" (UID: \"bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:12.964654 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.964612    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-scbkv\" (UID: \"bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:12.964816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.964667    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blg74\" (UniqueName: \"kubernetes.io/projected/bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb-kube-api-access-blg74\") pod \"kuadrant-operator-controller-manager-86878c769f-scbkv\" (UID: \"bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:12.965087 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.965064    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-scbkv\" (UID: \"bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:12.972711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:12.972692    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blg74\" (UniqueName: \"kubernetes.io/projected/bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb-kube-api-access-blg74\") pod \"kuadrant-operator-controller-manager-86878c769f-scbkv\" (UID: \"bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:13.037871 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:13.037806    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:13.384735 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:13.384711    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"]
Jul 25 01:34:13.386765 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:34:13.386741    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfb55f24_5f14_4b8c_b3b6_0aa2b7f785bb.slice/crio-d5a056906a3e1f41d626da50a09f3e33830e7096e54147d7e46cdfd0a4cc8662 WatchSource:0}: Error finding container d5a056906a3e1f41d626da50a09f3e33830e7096e54147d7e46cdfd0a4cc8662: Status 404 returned error can't find the container with id d5a056906a3e1f41d626da50a09f3e33830e7096e54147d7e46cdfd0a4cc8662
Jul 25 01:34:14.050154 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:14.050124    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv" event={"ID":"bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb","Type":"ContainerStarted","Data":"d9cc440317b9158b605d6d98e23b8839830b23c006f83d7728991667a77599ed"}
Jul 25 01:34:14.050535 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:14.050177    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:34:14.050535 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:14.050195    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv" event={"ID":"bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb","Type":"ContainerStarted","Data":"d5a056906a3e1f41d626da50a09f3e33830e7096e54147d7e46cdfd0a4cc8662"}
Jul 25 01:34:14.072922 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:14.072867    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv" podStartSLOduration=2.072848648 podStartE2EDuration="2.072848648s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:34:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:34:14.069043889 +0000 UTC m=+1332.540000228" watchObservedRunningTime="2026-07-25 01:34:14.072848648 +0000 UTC m=+1332.543804987"
Jul 25 01:34:17.031083 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:17.031010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:34:25.057386 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:34:25.057314    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-scbkv"
Jul 25 01:37:00.961773 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:37:00.961739    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:37:13.996816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:37:13.996783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:37:36.614552 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:37:36.614474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:37:37.499483 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:37:37.499449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:37:52.738900 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:37:52.738861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:40:52.132762 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:40:52.132727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:11.763692 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:11.763659    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:15.174899 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:15.174865    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:15.277023 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:15.276996    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:15.688897 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:15.688867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:21.397700 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:21.397629    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:21.497484 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:21.497449    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:21.651027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:21.650923    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:21.939456 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:21.939369    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"]
Jul 25 01:41:21.943425 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:21.943402    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:21.945797 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:21.945772    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-02d4b958-openshift-default-dockercfg-ps8xp\""
Jul 25 01:41:21.963584 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:21.963555    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:21.964326 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:21.964306    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"]
Jul 25 01:41:22.061019 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.060958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-credential-socket\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.061019 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.061015    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-certs\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.061241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.061055    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-envoy\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.061241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.061116    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-podinfo\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.061241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.061140    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-data\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.061241 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.061179    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-token\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.061389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.061257    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istiod-ca-cert\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.061389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.061309    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdjtb\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-kube-api-access-qdjtb\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.061389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.061338    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-socket\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.094224 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.094192    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:22.162404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162376    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-credential-socket\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162406    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-certs\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162422    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-envoy\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162455    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-podinfo\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162483    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-data\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162579    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-token\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162626    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istiod-ca-cert\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162909 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdjtb\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-kube-api-access-qdjtb\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162909 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-socket\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.162909 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162854    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-credential-socket\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.163088 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.162924    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-data\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.163088 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.163037    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-certs\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.163088 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.163048    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-socket\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.163414 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.163398    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istiod-ca-cert\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.164834 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.164813    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-envoy\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.164957 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.164941    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-podinfo\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.171899 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.171875    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdjtb\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-kube-api-access-qdjtb\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.172050 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.172026    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-token\") pod \"e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") " pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.253895 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.253833    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:22.396041 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.396019    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"]
Jul 25 01:41:22.397637 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:41:22.397617    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd65b7dfa_a468_437d_a26c_11e74a31b7b5.slice/crio-719165ad4d4e299c8ea767e59f16f07a41ae0bcdbeb8e618d7953b03077f294f WatchSource:0}: Error finding container 719165ad4d4e299c8ea767e59f16f07a41ae0bcdbeb8e618d7953b03077f294f: Status 404 returned error can't find the container with id 719165ad4d4e299c8ea767e59f16f07a41ae0bcdbeb8e618d7953b03077f294f
Jul 25 01:41:22.399587 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.399557    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:41:22.399885 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.399619    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:41:22.399885 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:22.399646    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:41:23.067547 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:23.067515    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" event={"ID":"d65b7dfa-a468-437d-a26c-11e74a31b7b5","Type":"ContainerStarted","Data":"a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f"}
Jul 25 01:41:23.067781 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:23.067604    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" event={"ID":"d65b7dfa-a468-437d-a26c-11e74a31b7b5","Type":"ContainerStarted","Data":"719165ad4d4e299c8ea767e59f16f07a41ae0bcdbeb8e618d7953b03077f294f"}
Jul 25 01:41:23.089232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:23.089181    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" podStartSLOduration=2.08916708 podStartE2EDuration="2.08916708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:41:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:41:23.086525585 +0000 UTC m=+1761.557481924" watchObservedRunningTime="2026-07-25 01:41:23.08916708 +0000 UTC m=+1761.560123417"
Jul 25 01:41:23.254164 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:23.254134    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:23.258907 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:23.258886    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:24.075869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:24.075839    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:24.076247 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:24.075908    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:24.431417 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:24.431362    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:24.515296 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:24.515260    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:24.588822 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:24.588795    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:25.088220 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.088191    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:25.174404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.174371    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:25.177940 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.177916    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"]
Jul 25 01:41:25.181589 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.181574    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:25.183906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.183887    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-02d4b958\""
Jul 25 01:41:25.191931 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.191913    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"]
Jul 25 01:41:25.208607 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.208579    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"]
Jul 25 01:41:25.211990 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.211966    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:25.214237 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.214222    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-02d4b958-dockercfg-k7v79\""
Jul 25 01:41:25.222873 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.222854    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"]
Jul 25 01:41:25.288950 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.288922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d9c22b-5401-41c6-8796-5fe6574f009f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw\" (UID: \"b1d9c22b-5401-41c6-8796-5fe6574f009f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:25.289097 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.289015    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8n9z2\" (UniqueName: \"kubernetes.io/projected/b1d9c22b-5401-41c6-8796-5fe6574f009f-kube-api-access-8n9z2\") pod \"payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw\" (UID: \"b1d9c22b-5401-41c6-8796-5fe6574f009f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:25.390349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.390314    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8n9z2\" (UniqueName: \"kubernetes.io/projected/b1d9c22b-5401-41c6-8796-5fe6574f009f-kube-api-access-8n9z2\") pod \"payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw\" (UID: \"b1d9c22b-5401-41c6-8796-5fe6574f009f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:25.390528 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.390416    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d9c22b-5401-41c6-8796-5fe6574f009f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw\" (UID: \"b1d9c22b-5401-41c6-8796-5fe6574f009f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:25.390528 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.390449    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-plugins-config-volume\") pod \"payload-processing-e2e-ait-02d4b958-98444579b-xp6g8\" (UID: \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\") " pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:25.390528 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.390474    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmg4h\" (UniqueName: \"kubernetes.io/projected/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-kube-api-access-wmg4h\") pod \"payload-processing-e2e-ait-02d4b958-98444579b-xp6g8\" (UID: \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\") " pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:25.391184 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.391162    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d9c22b-5401-41c6-8796-5fe6574f009f-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw\" (UID: \"b1d9c22b-5401-41c6-8796-5fe6574f009f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:25.398188 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.398167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8n9z2\" (UniqueName: \"kubernetes.io/projected/b1d9c22b-5401-41c6-8796-5fe6574f009f-kube-api-access-8n9z2\") pod \"payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw\" (UID: \"b1d9c22b-5401-41c6-8796-5fe6574f009f\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:25.491722 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.491695    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-plugins-config-volume\") pod \"payload-processing-e2e-ait-02d4b958-98444579b-xp6g8\" (UID: \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\") " pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:25.491858 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.491731    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmg4h\" (UniqueName: \"kubernetes.io/projected/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-kube-api-access-wmg4h\") pod \"payload-processing-e2e-ait-02d4b958-98444579b-xp6g8\" (UID: \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\") " pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:25.492260 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.492239    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:25.492649 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.492622    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-plugins-config-volume\") pod \"payload-processing-e2e-ait-02d4b958-98444579b-xp6g8\" (UID: \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\") " pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:25.499401 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.499381    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmg4h\" (UniqueName: \"kubernetes.io/projected/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-kube-api-access-wmg4h\") pod \"payload-processing-e2e-ait-02d4b958-98444579b-xp6g8\" (UID: \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\") " pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:25.523224 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.523154    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:25.652126 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.652097    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"]
Jul 25 01:41:25.652876 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:41:25.652842    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1d9c22b_5401_41c6_8796_5fe6574f009f.slice/crio-a87447637bf49f29c2afe7a419acdcd732eb57bf6f1aca0e7460bfb88dd3a4c4 WatchSource:0}: Error finding container a87447637bf49f29c2afe7a419acdcd732eb57bf6f1aca0e7460bfb88dd3a4c4: Status 404 returned error can't find the container with id a87447637bf49f29c2afe7a419acdcd732eb57bf6f1aca0e7460bfb88dd3a4c4
Jul 25 01:41:25.886837 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:25.886810    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"]
Jul 25 01:41:25.888319 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:41:25.888284    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a59ca17_9be1_44ca_8228_ba6dd90a1e5c.slice/crio-869901f0936a84f9bb2b3a282030f6218c8b62bab9c90258e0f8f4efd1d8fc69 WatchSource:0}: Error finding container 869901f0936a84f9bb2b3a282030f6218c8b62bab9c90258e0f8f4efd1d8fc69: Status 404 returned error can't find the container with id 869901f0936a84f9bb2b3a282030f6218c8b62bab9c90258e0f8f4efd1d8fc69
Jul 25 01:41:26.083424 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:26.083386    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw" event={"ID":"b1d9c22b-5401-41c6-8796-5fe6574f009f","Type":"ContainerStarted","Data":"5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556"}
Jul 25 01:41:26.083627 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:26.083442    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw" event={"ID":"b1d9c22b-5401-41c6-8796-5fe6574f009f","Type":"ContainerStarted","Data":"a87447637bf49f29c2afe7a419acdcd732eb57bf6f1aca0e7460bfb88dd3a4c4"}
Jul 25 01:41:26.083627 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:26.083546    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:26.084834 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:26.084811    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8" event={"ID":"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c","Type":"ContainerStarted","Data":"00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2"}
Jul 25 01:41:26.084935 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:26.084855    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:26.084935 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:26.084872    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8" event={"ID":"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c","Type":"ContainerStarted","Data":"869901f0936a84f9bb2b3a282030f6218c8b62bab9c90258e0f8f4efd1d8fc69"}
Jul 25 01:41:26.101232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:26.101173    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw" podStartSLOduration=1.101156295 podStartE2EDuration="1.101156295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:41:26.097963417 +0000 UTC m=+1764.568919784" watchObservedRunningTime="2026-07-25 01:41:26.101156295 +0000 UTC m=+1764.572112638"
Jul 25 01:41:26.114308 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:26.114270    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8" podStartSLOduration=1.114259409 podStartE2EDuration="1.114259409s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:41:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:41:26.11260263 +0000 UTC m=+1764.583558993" watchObservedRunningTime="2026-07-25 01:41:26.114259409 +0000 UTC m=+1764.585215795"
Jul 25 01:41:37.092508 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:37.092471    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:41:37.092896 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:37.092557    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:41:39.074641 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:39.074580    2562 patch_prober.go:28] interesting pod/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 25 01:41:39.075055 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:39.074661    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 25 01:41:39.151802 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:39.151774    2562 generic.go:356] "Generic (PLEG): container finished" podID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerID="a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f" exitCode=0
Jul 25 01:41:39.151952 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:39.151807    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" event={"ID":"d65b7dfa-a468-437d-a26c-11e74a31b7b5","Type":"ContainerDied","Data":"a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f"}
Jul 25 01:41:39.152208 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:39.152193    2562 scope.go:117] "RemoveContainer" containerID="a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f"
Jul 25 01:41:39.153328 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:39.153301    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:41:39.153404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:39.153383    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:41:39.153445 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:39.153418    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:41:40.163174 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:40.163124    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" event={"ID":"d65b7dfa-a468-437d-a26c-11e74a31b7b5","Type":"ContainerStarted","Data":"ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b"}
Jul 25 01:41:40.255316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:40.254833    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:40.255316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:40.254958    2562 patch_prober.go:28] interesting pod/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 25 01:41:40.255316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:40.255027    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 25 01:41:41.255226 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:41.255184    2562 patch_prober.go:28] interesting pod/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 25 01:41:41.255832 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:41.255801    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 25 01:41:42.254699 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:42.254664    2562 patch_prober.go:28] interesting pod/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 25 01:41:42.255014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:42.254961    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 25 01:41:43.255038 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:43.255003    2562 patch_prober.go:28] interesting pod/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body=
Jul 25 01:41:43.255608 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:43.255578    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused"
Jul 25 01:41:44.259271 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:44.259247    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:44.259660 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:44.259643    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:44.260248 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:44.260231    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:41:46.632747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:46.632718    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:41:46.681258 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:41:46.681223    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:00.692617 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:00.692580    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:00.711409 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:00.711375    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"]
Jul 25 01:42:00.711662 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:00.711635    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8" podUID="9a59ca17-9be1-44ca-8228-ba6dd90a1e5c" containerName="payload-processing" containerID="cri-o://00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2" gracePeriod=30
Jul 25 01:42:00.718958 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:00.718916    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"]
Jul 25 01:42:00.719412 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:00.719360    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw" podUID="b1d9c22b-5401-41c6-8796-5fe6574f009f" containerName="payload-pre-processing" containerID="cri-o://5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556" gracePeriod=30
Jul 25 01:42:00.749578 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:00.749548    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:00.806507 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:00.806484    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:00.977504 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:00.977478    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:42:01.037938 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.037911    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:42:01.123753 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.123729    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8n9z2\" (UniqueName: \"kubernetes.io/projected/b1d9c22b-5401-41c6-8796-5fe6574f009f-kube-api-access-8n9z2\") pod \"b1d9c22b-5401-41c6-8796-5fe6574f009f\" (UID: \"b1d9c22b-5401-41c6-8796-5fe6574f009f\") "
Jul 25 01:42:01.123902 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.123796    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-plugins-config-volume\") pod \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\" (UID: \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\") "
Jul 25 01:42:01.123902 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.123842    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d9c22b-5401-41c6-8796-5fe6574f009f-plugins-config-volume\") pod \"b1d9c22b-5401-41c6-8796-5fe6574f009f\" (UID: \"b1d9c22b-5401-41c6-8796-5fe6574f009f\") "
Jul 25 01:42:01.123902 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.123871    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmg4h\" (UniqueName: \"kubernetes.io/projected/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-kube-api-access-wmg4h\") pod \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\" (UID: \"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c\") "
Jul 25 01:42:01.124218 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.124189    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9a59ca17-9be1-44ca-8228-ba6dd90a1e5c" (UID: "9a59ca17-9be1-44ca-8228-ba6dd90a1e5c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:42:01.124316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.124220    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1d9c22b-5401-41c6-8796-5fe6574f009f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b1d9c22b-5401-41c6-8796-5fe6574f009f" (UID: "b1d9c22b-5401-41c6-8796-5fe6574f009f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:42:01.125784 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.125760    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1d9c22b-5401-41c6-8796-5fe6574f009f-kube-api-access-8n9z2" (OuterVolumeSpecName: "kube-api-access-8n9z2") pod "b1d9c22b-5401-41c6-8796-5fe6574f009f" (UID: "b1d9c22b-5401-41c6-8796-5fe6574f009f"). InnerVolumeSpecName "kube-api-access-8n9z2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:42:01.125845 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.125787    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-kube-api-access-wmg4h" (OuterVolumeSpecName: "kube-api-access-wmg4h") pod "9a59ca17-9be1-44ca-8228-ba6dd90a1e5c" (UID: "9a59ca17-9be1-44ca-8228-ba6dd90a1e5c"). InnerVolumeSpecName "kube-api-access-wmg4h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:42:01.225407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.225339    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:01.225407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.225360    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b1d9c22b-5401-41c6-8796-5fe6574f009f-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:01.225407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.225370    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmg4h\" (UniqueName: \"kubernetes.io/projected/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c-kube-api-access-wmg4h\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:01.225407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.225379    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8n9z2\" (UniqueName: \"kubernetes.io/projected/b1d9c22b-5401-41c6-8796-5fe6574f009f-kube-api-access-8n9z2\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:01.264709 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.264680    2562 generic.go:356] "Generic (PLEG): container finished" podID="b1d9c22b-5401-41c6-8796-5fe6574f009f" containerID="5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556" exitCode=0
Jul 25 01:42:01.264844 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.264724    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"
Jul 25 01:42:01.264844 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.264767    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw" event={"ID":"b1d9c22b-5401-41c6-8796-5fe6574f009f","Type":"ContainerDied","Data":"5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556"}
Jul 25 01:42:01.264844 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.264827    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw" event={"ID":"b1d9c22b-5401-41c6-8796-5fe6574f009f","Type":"ContainerDied","Data":"a87447637bf49f29c2afe7a419acdcd732eb57bf6f1aca0e7460bfb88dd3a4c4"}
Jul 25 01:42:01.265000 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.264860    2562 scope.go:117] "RemoveContainer" containerID="5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556"
Jul 25 01:42:01.266022 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.265999    2562 generic.go:356] "Generic (PLEG): container finished" podID="9a59ca17-9be1-44ca-8228-ba6dd90a1e5c" containerID="00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2" exitCode=0
Jul 25 01:42:01.266096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.266037    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"
Jul 25 01:42:01.266096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.266082    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8" event={"ID":"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c","Type":"ContainerDied","Data":"00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2"}
Jul 25 01:42:01.266218 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.266121    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8" event={"ID":"9a59ca17-9be1-44ca-8228-ba6dd90a1e5c","Type":"ContainerDied","Data":"869901f0936a84f9bb2b3a282030f6218c8b62bab9c90258e0f8f4efd1d8fc69"}
Jul 25 01:42:01.274682 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.274661    2562 scope.go:117] "RemoveContainer" containerID="5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556"
Jul 25 01:42:01.275084 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:42:01.275057    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556\": container with ID starting with 5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556 not found: ID does not exist" containerID="5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556"
Jul 25 01:42:01.275199 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.275095    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556"} err="failed to get container status \"5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556\": rpc error: code = NotFound desc = could not find container \"5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556\": container with ID starting with 5831f729f24caa3b0c7bbd2b2e1bf8ca5a8604e34982efdd54047f3ed01f8556 not found: ID does not exist"
Jul 25 01:42:01.275199 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.275117    2562 scope.go:117] "RemoveContainer" containerID="00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2"
Jul 25 01:42:01.283989 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.283963    2562 scope.go:117] "RemoveContainer" containerID="00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2"
Jul 25 01:42:01.284261 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:42:01.284243    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2\": container with ID starting with 00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2 not found: ID does not exist" containerID="00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2"
Jul 25 01:42:01.284327 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.284267    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2"} err="failed to get container status \"00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2\": rpc error: code = NotFound desc = could not find container \"00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2\": container with ID starting with 00144abb33ad19d4e3cce5be3d57e5183ab9fc4c4d2f7f529cb10c754fad8cd2 not found: ID does not exist"
Jul 25 01:42:01.291410 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.291389    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"]
Jul 25 01:42:01.295306 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.295286    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-02d4b958-98444579b-xp6g8"]
Jul 25 01:42:01.303911 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.303893    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"]
Jul 25 01:42:01.307702 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:01.307680    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-02d4b958-6f6f9f4d5b-vpjdw"]
Jul 25 01:42:02.076324 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:02.076259    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a59ca17-9be1-44ca-8228-ba6dd90a1e5c" path="/var/lib/kubelet/pods/9a59ca17-9be1-44ca-8228-ba6dd90a1e5c/volumes"
Jul 25 01:42:02.076652 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:02.076607    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1d9c22b-5401-41c6-8796-5fe6574f009f" path="/var/lib/kubelet/pods/b1d9c22b-5401-41c6-8796-5fe6574f009f/volumes"
Jul 25 01:42:05.146869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:05.146840    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:05.283512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:05.283476    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"]
Jul 25 01:42:05.283739 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:05.283712    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy" containerID="cri-o://ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b" gracePeriod=30
Jul 25 01:42:05.302311 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:05.302272    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:05.527883 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:05.527787    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:05.785257 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:05.785161    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:05.878944 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:05.878905    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:06.011710 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:06.011681    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:06.358090 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:06.358062    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:06.568167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:06.568137    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:10.543914 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.543841    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:42:10.610364 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610333    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qdjtb\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-kube-api-access-qdjtb\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.610364 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610365    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-credential-socket\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.610566 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610402    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-envoy\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.610566 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610522    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-certs\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.610566 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610551    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-podinfo\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.610728 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610618    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-token\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.610728 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610637    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:42:10.610728 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610657    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-data\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.610879 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610751    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-socket\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.610879 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610788    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:42:10.610879 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.610848    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istiod-ca-cert\") pod \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\" (UID: \"d65b7dfa-a468-437d-a26c-11e74a31b7b5\") "
Jul 25 01:42:10.611129 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.611084    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:42:10.611210 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.611170    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:10.611210 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.611191    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-credential-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:10.611210 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.611206    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-workload-certs\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:10.611424 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.611405    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:42:10.611663 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.611638    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-data" (OuterVolumeSpecName: "istio-data") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:42:10.612767 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.612739    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:42:10.612767 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.612747    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-token" (OuterVolumeSpecName: "istio-token") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:42:10.612964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.612943    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:42:10.613037 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.612949    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-kube-api-access-qdjtb" (OuterVolumeSpecName: "kube-api-access-qdjtb") pod "d65b7dfa-a468-437d-a26c-11e74a31b7b5" (UID: "d65b7dfa-a468-437d-a26c-11e74a31b7b5"). InnerVolumeSpecName "kube-api-access-qdjtb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:42:10.712286 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.712255    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-token\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:10.712286 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.712281    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-data\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:10.712286 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.712289    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istiod-ca-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:10.712513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.712299    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdjtb\" (UniqueName: \"kubernetes.io/projected/d65b7dfa-a468-437d-a26c-11e74a31b7b5-kube-api-access-qdjtb\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:10.712513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.712309    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-envoy\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:10.712513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:10.712317    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d65b7dfa-a468-437d-a26c-11e74a31b7b5-istio-podinfo\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:11.318151 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.318115    2562 generic.go:356] "Generic (PLEG): container finished" podID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerID="ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b" exitCode=0
Jul 25 01:42:11.318322 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.318157    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" event={"ID":"d65b7dfa-a468-437d-a26c-11e74a31b7b5","Type":"ContainerDied","Data":"ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b"}
Jul 25 01:42:11.318322 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.318167    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"
Jul 25 01:42:11.318322 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.318207    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm" event={"ID":"d65b7dfa-a468-437d-a26c-11e74a31b7b5","Type":"ContainerDied","Data":"719165ad4d4e299c8ea767e59f16f07a41ae0bcdbeb8e618d7953b03077f294f"}
Jul 25 01:42:11.318322 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.318244    2562 scope.go:117] "RemoveContainer" containerID="ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b"
Jul 25 01:42:11.327396 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.327380    2562 scope.go:117] "RemoveContainer" containerID="a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f"
Jul 25 01:42:11.334853 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.334829    2562 scope.go:117] "RemoveContainer" containerID="ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b"
Jul 25 01:42:11.335109 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:42:11.335093    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b\": container with ID starting with ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b not found: ID does not exist" containerID="ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b"
Jul 25 01:42:11.335181 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.335115    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b"} err="failed to get container status \"ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b\": rpc error: code = NotFound desc = could not find container \"ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b\": container with ID starting with ae237f4ddb7770d834134c6306e844c47863a82e24d0e1f21b79b64877a0364b not found: ID does not exist"
Jul 25 01:42:11.335181 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.335129    2562 scope.go:117] "RemoveContainer" containerID="a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f"
Jul 25 01:42:11.335588 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:42:11.335570    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f\": container with ID starting with a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f not found: ID does not exist" containerID="a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f"
Jul 25 01:42:11.335636 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.335593    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f"} err="failed to get container status \"a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f\": rpc error: code = NotFound desc = could not find container \"a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f\": container with ID starting with a4e8d2638e4adb6cdfe9489f574c430e0008a7286dca2af033ca161b1914a32f not found: ID does not exist"
Jul 25 01:42:11.339957 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.339938    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"]
Jul 25 01:42:11.344942 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:11.344921    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-02d4b958-openshift-default-849655f9d4-mqgcm"]
Jul 25 01:42:12.079923 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:12.079891    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" path="/var/lib/kubelet/pods/d65b7dfa-a468-437d-a26c-11e74a31b7b5/volumes"
Jul 25 01:42:12.285054 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:12.285023    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:12.395997 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:12.395893    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:12.627485 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:12.627453    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:12.982081 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:12.982042    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:13.086942 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.086912    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:13.087910 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.087890    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"]
Jul 25 01:42:13.088269 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088256    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a59ca17-9be1-44ca-8228-ba6dd90a1e5c" containerName="payload-processing"
Jul 25 01:42:13.088312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088270    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a59ca17-9be1-44ca-8228-ba6dd90a1e5c" containerName="payload-processing"
Jul 25 01:42:13.088312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088281    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy"
Jul 25 01:42:13.088312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088287    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy"
Jul 25 01:42:13.088312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088300    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1d9c22b-5401-41c6-8796-5fe6574f009f" containerName="payload-pre-processing"
Jul 25 01:42:13.088312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088305    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1d9c22b-5401-41c6-8796-5fe6574f009f" containerName="payload-pre-processing"
Jul 25 01:42:13.088466 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088317    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy"
Jul 25 01:42:13.088466 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088322    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy"
Jul 25 01:42:13.088466 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088380    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a59ca17-9be1-44ca-8228-ba6dd90a1e5c" containerName="payload-processing"
Jul 25 01:42:13.088466 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088387    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy"
Jul 25 01:42:13.088466 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088396    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="d65b7dfa-a468-437d-a26c-11e74a31b7b5" containerName="istio-proxy"
Jul 25 01:42:13.088466 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.088402    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1d9c22b-5401-41c6-8796-5fe6574f009f" containerName="payload-pre-processing"
Jul 25 01:42:13.092847 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.092832    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:13.095206 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.095188    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-842d4926\""
Jul 25 01:42:13.095335 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.095191    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-842d4926-dockercfg-rlqwb\""
Jul 25 01:42:13.103782 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.103748    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"]
Jul 25 01:42:13.119738 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.119710    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"]
Jul 25 01:42:13.123275 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.123255    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:13.133207 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.133185    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"]
Jul 25 01:42:13.137276 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.137253    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzwcn\" (UniqueName: \"kubernetes.io/projected/57e48579-27bd-4867-bda2-68fdfd6f4eeb-kube-api-access-bzwcn\") pod \"payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd\" (UID: \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:13.137517 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.137490    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e48579-27bd-4867-bda2-68fdfd6f4eeb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd\" (UID: \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:13.137641 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.137537    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zg5hh\" (UniqueName: \"kubernetes.io/projected/79cc4243-053b-412c-aab6-b4803ec6e868-kube-api-access-zg5hh\") pod \"payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z\" (UID: \"79cc4243-053b-412c-aab6-b4803ec6e868\") " pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:13.137641 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.137563    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79cc4243-053b-412c-aab6-b4803ec6e868-plugins-config-volume\") pod \"payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z\" (UID: \"79cc4243-053b-412c-aab6-b4803ec6e868\") " pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:13.238478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.238384    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e48579-27bd-4867-bda2-68fdfd6f4eeb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd\" (UID: \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:13.238478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.238426    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zg5hh\" (UniqueName: \"kubernetes.io/projected/79cc4243-053b-412c-aab6-b4803ec6e868-kube-api-access-zg5hh\") pod \"payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z\" (UID: \"79cc4243-053b-412c-aab6-b4803ec6e868\") " pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:13.238478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.238452    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79cc4243-053b-412c-aab6-b4803ec6e868-plugins-config-volume\") pod \"payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z\" (UID: \"79cc4243-053b-412c-aab6-b4803ec6e868\") " pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:13.238753 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.238523    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzwcn\" (UniqueName: \"kubernetes.io/projected/57e48579-27bd-4867-bda2-68fdfd6f4eeb-kube-api-access-bzwcn\") pod \"payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd\" (UID: \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:13.239652 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.239626    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e48579-27bd-4867-bda2-68fdfd6f4eeb-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd\" (UID: \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:13.239652 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.239647    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79cc4243-053b-412c-aab6-b4803ec6e868-plugins-config-volume\") pod \"payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z\" (UID: \"79cc4243-053b-412c-aab6-b4803ec6e868\") " pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:13.246149 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.246122    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zg5hh\" (UniqueName: \"kubernetes.io/projected/79cc4243-053b-412c-aab6-b4803ec6e868-kube-api-access-zg5hh\") pod \"payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z\" (UID: \"79cc4243-053b-412c-aab6-b4803ec6e868\") " pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:13.246271 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.246236    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzwcn\" (UniqueName: \"kubernetes.io/projected/57e48579-27bd-4867-bda2-68fdfd6f4eeb-kube-api-access-bzwcn\") pod \"payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd\" (UID: \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:13.404677 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.404644    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:13.436227 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.436197    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:13.558648 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.558616    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"]
Jul 25 01:42:13.561649 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:42:13.561613    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57e48579_27bd_4867_bda2_68fdfd6f4eeb.slice/crio-98a6f00ab2266a01151a1fa9839578df7127e985b841691b92ba69b7525f60ad WatchSource:0}: Error finding container 98a6f00ab2266a01151a1fa9839578df7127e985b841691b92ba69b7525f60ad: Status 404 returned error can't find the container with id 98a6f00ab2266a01151a1fa9839578df7127e985b841691b92ba69b7525f60ad
Jul 25 01:42:13.796568 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:13.796541    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"]
Jul 25 01:42:13.798291 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:42:13.798268    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79cc4243_053b_412c_aab6_b4803ec6e868.slice/crio-ea8f70d8ae382674586a010fb16edf664987546a1469a4d626201953c9d782b6 WatchSource:0}: Error finding container ea8f70d8ae382674586a010fb16edf664987546a1469a4d626201953c9d782b6: Status 404 returned error can't find the container with id ea8f70d8ae382674586a010fb16edf664987546a1469a4d626201953c9d782b6
Jul 25 01:42:14.335967 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:14.335935    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd" event={"ID":"57e48579-27bd-4867-bda2-68fdfd6f4eeb","Type":"ContainerStarted","Data":"930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80"}
Jul 25 01:42:14.336450 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:14.336002    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd" event={"ID":"57e48579-27bd-4867-bda2-68fdfd6f4eeb","Type":"ContainerStarted","Data":"98a6f00ab2266a01151a1fa9839578df7127e985b841691b92ba69b7525f60ad"}
Jul 25 01:42:14.336450 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:14.336032    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:14.337618 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:14.337581    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z" event={"ID":"79cc4243-053b-412c-aab6-b4803ec6e868","Type":"ContainerStarted","Data":"0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c"}
Jul 25 01:42:14.337756 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:14.337644    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z" event={"ID":"79cc4243-053b-412c-aab6-b4803ec6e868","Type":"ContainerStarted","Data":"ea8f70d8ae382674586a010fb16edf664987546a1469a4d626201953c9d782b6"}
Jul 25 01:42:14.337756 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:14.337676    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:14.351073 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:14.351029    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd" podStartSLOduration=1.3510141789999999 podStartE2EDuration="1.351014179s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:42:14.350226966 +0000 UTC m=+1812.821183304" watchObservedRunningTime="2026-07-25 01:42:14.351014179 +0000 UTC m=+1812.821970516"
Jul 25 01:42:14.367179 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:14.367128    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z" podStartSLOduration=1.3671103709999999 podStartE2EDuration="1.367110371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:42:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:42:14.366619002 +0000 UTC m=+1812.837575341" watchObservedRunningTime="2026-07-25 01:42:14.367110371 +0000 UTC m=+1812.838066709"
Jul 25 01:42:25.347093 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:25.347052    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:25.347535 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:25.347515    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:47.742673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:47.742642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:47.771258 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:47.771227    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"]
Jul 25 01:42:47.771496 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:47.771464    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z" podUID="79cc4243-053b-412c-aab6-b4803ec6e868" containerName="payload-processing" containerID="cri-o://0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c" gracePeriod=30
Jul 25 01:42:47.778379 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:47.778343    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"]
Jul 25 01:42:47.778609 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:47.778563    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd" podUID="57e48579-27bd-4867-bda2-68fdfd6f4eeb" containerName="payload-pre-processing" containerID="cri-o://930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80" gracePeriod=30
Jul 25 01:42:47.811493 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:47.811465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:47.867850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:47.867823    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:48.030102 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.030080    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:48.062215 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.062193    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:48.129544 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.129521    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zg5hh\" (UniqueName: \"kubernetes.io/projected/79cc4243-053b-412c-aab6-b4803ec6e868-kube-api-access-zg5hh\") pod \"79cc4243-053b-412c-aab6-b4803ec6e868\" (UID: \"79cc4243-053b-412c-aab6-b4803ec6e868\") "
Jul 25 01:42:48.129710 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.129567    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e48579-27bd-4867-bda2-68fdfd6f4eeb-plugins-config-volume\") pod \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\" (UID: \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\") "
Jul 25 01:42:48.129710 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.129619    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzwcn\" (UniqueName: \"kubernetes.io/projected/57e48579-27bd-4867-bda2-68fdfd6f4eeb-kube-api-access-bzwcn\") pod \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\" (UID: \"57e48579-27bd-4867-bda2-68fdfd6f4eeb\") "
Jul 25 01:42:48.129710 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.129650    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79cc4243-053b-412c-aab6-b4803ec6e868-plugins-config-volume\") pod \"79cc4243-053b-412c-aab6-b4803ec6e868\" (UID: \"79cc4243-053b-412c-aab6-b4803ec6e868\") "
Jul 25 01:42:48.130026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.130005    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/57e48579-27bd-4867-bda2-68fdfd6f4eeb-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "57e48579-27bd-4867-bda2-68fdfd6f4eeb" (UID: "57e48579-27bd-4867-bda2-68fdfd6f4eeb"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:42:48.130112 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.130071    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/79cc4243-053b-412c-aab6-b4803ec6e868-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "79cc4243-053b-412c-aab6-b4803ec6e868" (UID: "79cc4243-053b-412c-aab6-b4803ec6e868"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:42:48.131630 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.131604    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/79cc4243-053b-412c-aab6-b4803ec6e868-kube-api-access-zg5hh" (OuterVolumeSpecName: "kube-api-access-zg5hh") pod "79cc4243-053b-412c-aab6-b4803ec6e868" (UID: "79cc4243-053b-412c-aab6-b4803ec6e868"). InnerVolumeSpecName "kube-api-access-zg5hh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:42:48.131739 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.131714    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57e48579-27bd-4867-bda2-68fdfd6f4eeb-kube-api-access-bzwcn" (OuterVolumeSpecName: "kube-api-access-bzwcn") pod "57e48579-27bd-4867-bda2-68fdfd6f4eeb" (UID: "57e48579-27bd-4867-bda2-68fdfd6f4eeb"). InnerVolumeSpecName "kube-api-access-bzwcn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:42:48.231275 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.231249    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zg5hh\" (UniqueName: \"kubernetes.io/projected/79cc4243-053b-412c-aab6-b4803ec6e868-kube-api-access-zg5hh\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:48.231275 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.231271    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/57e48579-27bd-4867-bda2-68fdfd6f4eeb-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:48.231413 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.231281    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzwcn\" (UniqueName: \"kubernetes.io/projected/57e48579-27bd-4867-bda2-68fdfd6f4eeb-kube-api-access-bzwcn\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:48.231413 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.231289    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/79cc4243-053b-412c-aab6-b4803ec6e868-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:42:48.507885 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.507852    2562 generic.go:356] "Generic (PLEG): container finished" podID="57e48579-27bd-4867-bda2-68fdfd6f4eeb" containerID="930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80" exitCode=0
Jul 25 01:42:48.508088 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.507905    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"
Jul 25 01:42:48.508088 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.507937    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd" event={"ID":"57e48579-27bd-4867-bda2-68fdfd6f4eeb","Type":"ContainerDied","Data":"930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80"}
Jul 25 01:42:48.508088 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.508034    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd" event={"ID":"57e48579-27bd-4867-bda2-68fdfd6f4eeb","Type":"ContainerDied","Data":"98a6f00ab2266a01151a1fa9839578df7127e985b841691b92ba69b7525f60ad"}
Jul 25 01:42:48.508088 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.508058    2562 scope.go:117] "RemoveContainer" containerID="930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80"
Jul 25 01:42:48.509281 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.509255    2562 generic.go:356] "Generic (PLEG): container finished" podID="79cc4243-053b-412c-aab6-b4803ec6e868" containerID="0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c" exitCode=0
Jul 25 01:42:48.509407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.509317    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"
Jul 25 01:42:48.509407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.509326    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z" event={"ID":"79cc4243-053b-412c-aab6-b4803ec6e868","Type":"ContainerDied","Data":"0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c"}
Jul 25 01:42:48.509407 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.509369    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z" event={"ID":"79cc4243-053b-412c-aab6-b4803ec6e868","Type":"ContainerDied","Data":"ea8f70d8ae382674586a010fb16edf664987546a1469a4d626201953c9d782b6"}
Jul 25 01:42:48.520398 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.520225    2562 scope.go:117] "RemoveContainer" containerID="930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80"
Jul 25 01:42:48.520790 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:42:48.520752    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80\": container with ID starting with 930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80 not found: ID does not exist" containerID="930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80"
Jul 25 01:42:48.520883 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.520786    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80"} err="failed to get container status \"930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80\": rpc error: code = NotFound desc = could not find container \"930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80\": container with ID starting with 930ce19970884eae17fa0037b0ea9148a9d9ca6bbeb6dad8d925dcf9e2c66f80 not found: ID does not exist"
Jul 25 01:42:48.520883 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.520810    2562 scope.go:117] "RemoveContainer" containerID="0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c"
Jul 25 01:42:48.528476 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.528459    2562 scope.go:117] "RemoveContainer" containerID="0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c"
Jul 25 01:42:48.528727 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:42:48.528710    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c\": container with ID starting with 0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c not found: ID does not exist" containerID="0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c"
Jul 25 01:42:48.528775 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.528733    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c"} err="failed to get container status \"0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c\": rpc error: code = NotFound desc = could not find container \"0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c\": container with ID starting with 0975ab6feae1b520cbdf599617920691305b1df8e0d427e815bdd112f323e25c not found: ID does not exist"
Jul 25 01:42:48.533513 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.533492    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"]
Jul 25 01:42:48.537195 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.537176    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-842d4926-76d68b9669-slz7z"]
Jul 25 01:42:48.546328 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.546310    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"]
Jul 25 01:42:48.550406 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:48.550384    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-842d4926-5fc6f94544-rd5fd"]
Jul 25 01:42:50.076576 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:50.076546    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57e48579-27bd-4867-bda2-68fdfd6f4eeb" path="/var/lib/kubelet/pods/57e48579-27bd-4867-bda2-68fdfd6f4eeb/volumes"
Jul 25 01:42:50.076933 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:50.076911    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="79cc4243-053b-412c-aab6-b4803ec6e868" path="/var/lib/kubelet/pods/79cc4243-053b-412c-aab6-b4803ec6e868/volumes"
Jul 25 01:42:52.402966 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:52.402922    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:52.563505 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:52.563476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:52.799204 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:52.799118    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:53.054058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:53.053972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:53.364845 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:53.364769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:54.195509 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:54.195482    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:55.517014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:55.516969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:55.582834 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:55.582802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:55.732468 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:55.732443    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:55.968379 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:55.968343    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:56.268447 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.268365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:56.362076 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.362046    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:42:56.370260 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.370235    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"]
Jul 25 01:42:56.370695 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.370679    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57e48579-27bd-4867-bda2-68fdfd6f4eeb" containerName="payload-pre-processing"
Jul 25 01:42:56.370739 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.370700    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="57e48579-27bd-4867-bda2-68fdfd6f4eeb" containerName="payload-pre-processing"
Jul 25 01:42:56.370739 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.370717    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="79cc4243-053b-412c-aab6-b4803ec6e868" containerName="payload-processing"
Jul 25 01:42:56.370739 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.370728    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="79cc4243-053b-412c-aab6-b4803ec6e868" containerName="payload-processing"
Jul 25 01:42:56.370835 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.370818    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="79cc4243-053b-412c-aab6-b4803ec6e868" containerName="payload-processing"
Jul 25 01:42:56.370869 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.370836    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="57e48579-27bd-4867-bda2-68fdfd6f4eeb" containerName="payload-pre-processing"
Jul 25 01:42:56.374751 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.374732    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:56.377214 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.377195    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-193600d6\""
Jul 25 01:42:56.377331 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.377214    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-193600d6-dockercfg-kv9z4\""
Jul 25 01:42:56.382642 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.382616    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"]
Jul 25 01:42:56.400960 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.400941    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"]
Jul 25 01:42:56.404067 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.404048    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:56.412923 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.412902    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"]
Jul 25 01:42:56.507673 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.507640    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72jlt\" (UniqueName: \"kubernetes.io/projected/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-kube-api-access-72jlt\") pod \"payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb\" (UID: \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:56.507831 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.507725    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kcl6v\" (UniqueName: \"kubernetes.io/projected/3189a215-7fcf-4691-824c-24fa6f3a0419-kube-api-access-kcl6v\") pod \"payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7\" (UID: \"3189a215-7fcf-4691-824c-24fa6f3a0419\") " pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:56.507831 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.507758    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb\" (UID: \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:56.507831 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.507783    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189a215-7fcf-4691-824c-24fa6f3a0419-plugins-config-volume\") pod \"payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7\" (UID: \"3189a215-7fcf-4691-824c-24fa6f3a0419\") " pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:56.609219 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.609144    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb\" (UID: \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:56.609219 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.609185    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189a215-7fcf-4691-824c-24fa6f3a0419-plugins-config-volume\") pod \"payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7\" (UID: \"3189a215-7fcf-4691-824c-24fa6f3a0419\") " pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:56.609631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.609227    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72jlt\" (UniqueName: \"kubernetes.io/projected/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-kube-api-access-72jlt\") pod \"payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb\" (UID: \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:56.609631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.609300    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kcl6v\" (UniqueName: \"kubernetes.io/projected/3189a215-7fcf-4691-824c-24fa6f3a0419-kube-api-access-kcl6v\") pod \"payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7\" (UID: \"3189a215-7fcf-4691-824c-24fa6f3a0419\") " pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:56.609890 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.609873    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb\" (UID: \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:56.609951 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.609933    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189a215-7fcf-4691-824c-24fa6f3a0419-plugins-config-volume\") pod \"payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7\" (UID: \"3189a215-7fcf-4691-824c-24fa6f3a0419\") " pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:56.617137 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.617114    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-72jlt\" (UniqueName: \"kubernetes.io/projected/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-kube-api-access-72jlt\") pod \"payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb\" (UID: \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:56.617137 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.617126    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kcl6v\" (UniqueName: \"kubernetes.io/projected/3189a215-7fcf-4691-824c-24fa6f3a0419-kube-api-access-kcl6v\") pod \"payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7\" (UID: \"3189a215-7fcf-4691-824c-24fa6f3a0419\") " pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:56.686788 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.686761    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:56.714642 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.714618    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:56.861453 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:56.861415    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"]
Jul 25 01:42:56.863669 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:42:56.863645    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e9fb0c1_51f9_42f9_b41b_498aee0840e4.slice/crio-d8fbdf83a92cae2e570e939a9b775b58af3f17e9c532f9feb46293e342ed5bc8 WatchSource:0}: Error finding container d8fbdf83a92cae2e570e939a9b775b58af3f17e9c532f9feb46293e342ed5bc8: Status 404 returned error can't find the container with id d8fbdf83a92cae2e570e939a9b775b58af3f17e9c532f9feb46293e342ed5bc8
Jul 25 01:42:57.091749 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.091725    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"]
Jul 25 01:42:57.093634 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:42:57.093609    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3189a215_7fcf_4691_824c_24fa6f3a0419.slice/crio-d29712acb88462fc2d269125a05efe6fc24a7fa269ecf8d48b50390e086a1cb8 WatchSource:0}: Error finding container d29712acb88462fc2d269125a05efe6fc24a7fa269ecf8d48b50390e086a1cb8: Status 404 returned error can't find the container with id d29712acb88462fc2d269125a05efe6fc24a7fa269ecf8d48b50390e086a1cb8
Jul 25 01:42:57.568154 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.568068    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb" event={"ID":"4e9fb0c1-51f9-42f9-b41b-498aee0840e4","Type":"ContainerStarted","Data":"af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62"}
Jul 25 01:42:57.568154 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.568125    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb" event={"ID":"4e9fb0c1-51f9-42f9-b41b-498aee0840e4","Type":"ContainerStarted","Data":"d8fbdf83a92cae2e570e939a9b775b58af3f17e9c532f9feb46293e342ed5bc8"}
Jul 25 01:42:57.568359 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.568159    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:42:57.569617 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.569584    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7" event={"ID":"3189a215-7fcf-4691-824c-24fa6f3a0419","Type":"ContainerStarted","Data":"f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636"}
Jul 25 01:42:57.569747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.569638    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7" event={"ID":"3189a215-7fcf-4691-824c-24fa6f3a0419","Type":"ContainerStarted","Data":"d29712acb88462fc2d269125a05efe6fc24a7fa269ecf8d48b50390e086a1cb8"}
Jul 25 01:42:57.569747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.569667    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:42:57.583047 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.583000    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb" podStartSLOduration=1.582967558 podStartE2EDuration="1.582967558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:42:57.582872339 +0000 UTC m=+1856.053828678" watchObservedRunningTime="2026-07-25 01:42:57.582967558 +0000 UTC m=+1856.053923901"
Jul 25 01:42:57.598434 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:42:57.598395    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7" podStartSLOduration=1.598383761 podStartE2EDuration="1.598383761s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:42:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:42:57.596563109 +0000 UTC m=+1856.067519487" watchObservedRunningTime="2026-07-25 01:42:57.598383761 +0000 UTC m=+1856.069340098"
Jul 25 01:43:08.576396 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:08.576361    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:43:08.576969 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:08.576554    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:43:31.893902 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:31.893802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:31.915834 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:31.915801    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"]
Jul 25 01:43:31.916039 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:31.916015    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7" podUID="3189a215-7fcf-4691-824c-24fa6f3a0419" containerName="payload-processing" containerID="cri-o://f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636" gracePeriod=30
Jul 25 01:43:31.923622 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:31.923600    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"]
Jul 25 01:43:31.923846 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:31.923825    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb" podUID="4e9fb0c1-51f9-42f9-b41b-498aee0840e4" containerName="payload-pre-processing" containerID="cri-o://af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62" gracePeriod=30
Jul 25 01:43:31.953324 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:31.953282    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:32.017482 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.017438    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:32.209111 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.209088    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:43:32.211997 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.211958    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:43:32.319761 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.319736    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189a215-7fcf-4691-824c-24fa6f3a0419-plugins-config-volume\") pod \"3189a215-7fcf-4691-824c-24fa6f3a0419\" (UID: \"3189a215-7fcf-4691-824c-24fa6f3a0419\") "
Jul 25 01:43:32.319918 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.319829    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kcl6v\" (UniqueName: \"kubernetes.io/projected/3189a215-7fcf-4691-824c-24fa6f3a0419-kube-api-access-kcl6v\") pod \"3189a215-7fcf-4691-824c-24fa6f3a0419\" (UID: \"3189a215-7fcf-4691-824c-24fa6f3a0419\") "
Jul 25 01:43:32.319918 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.319849    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-plugins-config-volume\") pod \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\" (UID: \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\") "
Jul 25 01:43:32.319918 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.319876    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-72jlt\" (UniqueName: \"kubernetes.io/projected/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-kube-api-access-72jlt\") pod \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\" (UID: \"4e9fb0c1-51f9-42f9-b41b-498aee0840e4\") "
Jul 25 01:43:32.320170 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.320144    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3189a215-7fcf-4691-824c-24fa6f3a0419-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3189a215-7fcf-4691-824c-24fa6f3a0419" (UID: "3189a215-7fcf-4691-824c-24fa6f3a0419"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:43:32.320293 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.320204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4e9fb0c1-51f9-42f9-b41b-498aee0840e4" (UID: "4e9fb0c1-51f9-42f9-b41b-498aee0840e4"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:43:32.321887 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.321868    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3189a215-7fcf-4691-824c-24fa6f3a0419-kube-api-access-kcl6v" (OuterVolumeSpecName: "kube-api-access-kcl6v") pod "3189a215-7fcf-4691-824c-24fa6f3a0419" (UID: "3189a215-7fcf-4691-824c-24fa6f3a0419"). InnerVolumeSpecName "kube-api-access-kcl6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:43:32.321887 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.321876    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-kube-api-access-72jlt" (OuterVolumeSpecName: "kube-api-access-72jlt") pod "4e9fb0c1-51f9-42f9-b41b-498aee0840e4" (UID: "4e9fb0c1-51f9-42f9-b41b-498aee0840e4"). InnerVolumeSpecName "kube-api-access-72jlt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:43:32.421288 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.421263    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kcl6v\" (UniqueName: \"kubernetes.io/projected/3189a215-7fcf-4691-824c-24fa6f3a0419-kube-api-access-kcl6v\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:43:32.421288 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.421285    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:43:32.421478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.421299    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-72jlt\" (UniqueName: \"kubernetes.io/projected/4e9fb0c1-51f9-42f9-b41b-498aee0840e4-kube-api-access-72jlt\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:43:32.421478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.421313    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3189a215-7fcf-4691-824c-24fa6f3a0419-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:43:32.742314 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.742280    2562 generic.go:356] "Generic (PLEG): container finished" podID="3189a215-7fcf-4691-824c-24fa6f3a0419" containerID="f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636" exitCode=0
Jul 25 01:43:32.742499 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.742339    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"
Jul 25 01:43:32.742499 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.742367    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7" event={"ID":"3189a215-7fcf-4691-824c-24fa6f3a0419","Type":"ContainerDied","Data":"f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636"}
Jul 25 01:43:32.742499 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.742424    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7" event={"ID":"3189a215-7fcf-4691-824c-24fa6f3a0419","Type":"ContainerDied","Data":"d29712acb88462fc2d269125a05efe6fc24a7fa269ecf8d48b50390e086a1cb8"}
Jul 25 01:43:32.742499 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.742454    2562 scope.go:117] "RemoveContainer" containerID="f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636"
Jul 25 01:43:32.743926 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.743903    2562 generic.go:356] "Generic (PLEG): container finished" podID="4e9fb0c1-51f9-42f9-b41b-498aee0840e4" containerID="af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62" exitCode=0
Jul 25 01:43:32.744084 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.743932    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb" event={"ID":"4e9fb0c1-51f9-42f9-b41b-498aee0840e4","Type":"ContainerDied","Data":"af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62"}
Jul 25 01:43:32.744084 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.743949    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"
Jul 25 01:43:32.744084 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.743964    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb" event={"ID":"4e9fb0c1-51f9-42f9-b41b-498aee0840e4","Type":"ContainerDied","Data":"d8fbdf83a92cae2e570e939a9b775b58af3f17e9c532f9feb46293e342ed5bc8"}
Jul 25 01:43:32.753377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.753361    2562 scope.go:117] "RemoveContainer" containerID="f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636"
Jul 25 01:43:32.753686 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:43:32.753669    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636\": container with ID starting with f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636 not found: ID does not exist" containerID="f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636"
Jul 25 01:43:32.753776 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.753696    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636"} err="failed to get container status \"f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636\": rpc error: code = NotFound desc = could not find container \"f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636\": container with ID starting with f650367d798b54bfaebac3e387e13d7d86b97919a9b790d290ee60beecfae636 not found: ID does not exist"
Jul 25 01:43:32.753776 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.753719    2562 scope.go:117] "RemoveContainer" containerID="af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62"
Jul 25 01:43:32.761794 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.761777    2562 scope.go:117] "RemoveContainer" containerID="af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62"
Jul 25 01:43:32.762144 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:43:32.762126    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62\": container with ID starting with af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62 not found: ID does not exist" containerID="af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62"
Jul 25 01:43:32.762200 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.762150    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62"} err="failed to get container status \"af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62\": rpc error: code = NotFound desc = could not find container \"af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62\": container with ID starting with af96ea206b6d398f7c9882f112d4cb2f813cf388eb3b09fe7acf31c87957cd62 not found: ID does not exist"
Jul 25 01:43:32.767308 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.767258    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"]
Jul 25 01:43:32.769686 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.769662    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-193600d6-7f45485848-tmngb"]
Jul 25 01:43:32.778663 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.778645    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"]
Jul 25 01:43:32.782509 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:32.782489    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-193600d6-5fffb4c98-rlnz7"]
Jul 25 01:43:34.080972 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:34.080936    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3189a215-7fcf-4691-824c-24fa6f3a0419" path="/var/lib/kubelet/pods/3189a215-7fcf-4691-824c-24fa6f3a0419/volumes"
Jul 25 01:43:34.081344 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:34.081319    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e9fb0c1-51f9-42f9-b41b-498aee0840e4" path="/var/lib/kubelet/pods/4e9fb0c1-51f9-42f9-b41b-498aee0840e4/volumes"
Jul 25 01:43:36.435303 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:36.435272    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:37.715360 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:37.715325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:37.966711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:37.966628    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:38.206404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:38.206376    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:38.517764 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:38.517736    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:39.118985 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:39.118940    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:40.680589 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:40.680540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:40.761654 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:40.761616    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:40.834116 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:40.834085    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:41.254297 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.254254    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:41.361889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.361856    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"]
Jul 25 01:43:41.362272 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.362257    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e9fb0c1-51f9-42f9-b41b-498aee0840e4" containerName="payload-pre-processing"
Jul 25 01:43:41.362331 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.362273    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e9fb0c1-51f9-42f9-b41b-498aee0840e4" containerName="payload-pre-processing"
Jul 25 01:43:41.362331 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.362282    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3189a215-7fcf-4691-824c-24fa6f3a0419" containerName="payload-processing"
Jul 25 01:43:41.362331 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.362288    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="3189a215-7fcf-4691-824c-24fa6f3a0419" containerName="payload-processing"
Jul 25 01:43:41.362443 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.362351    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e9fb0c1-51f9-42f9-b41b-498aee0840e4" containerName="payload-pre-processing"
Jul 25 01:43:41.362443 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.362362    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="3189a215-7fcf-4691-824c-24fa6f3a0419" containerName="payload-processing"
Jul 25 01:43:41.364238 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.364205    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:41.365725 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.365710    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:41.368035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.368018    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-906a2829\""
Jul 25 01:43:41.368135 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.368019    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-906a2829-dockercfg-jzkdp\""
Jul 25 01:43:41.376219 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.376200    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"]
Jul 25 01:43:41.394444 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.394415    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"]
Jul 25 01:43:41.397899 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.397883    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:41.409059 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.409041    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"]
Jul 25 01:43:41.501118 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.501077    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl\" (UID: \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:41.501118 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.501121    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnpcf\" (UniqueName: \"kubernetes.io/projected/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-kube-api-access-xnpcf\") pod \"payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh\" (UID: \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\") " pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:41.501374 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.501142    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-plugins-config-volume\") pod \"payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh\" (UID: \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\") " pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:41.501374 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.501223    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csdgm\" (UniqueName: \"kubernetes.io/projected/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-kube-api-access-csdgm\") pod \"payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl\" (UID: \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:41.602871 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.602795    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnpcf\" (UniqueName: \"kubernetes.io/projected/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-kube-api-access-xnpcf\") pod \"payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh\" (UID: \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\") " pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:41.602871 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.602831    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-plugins-config-volume\") pod \"payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh\" (UID: \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\") " pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:41.603063 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.603031    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-csdgm\" (UniqueName: \"kubernetes.io/projected/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-kube-api-access-csdgm\") pod \"payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl\" (UID: \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:41.603191 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.603172    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl\" (UID: \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:41.603457 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.603442    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-plugins-config-volume\") pod \"payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh\" (UID: \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\") " pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:41.603755 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.603740    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl\" (UID: \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:41.610299 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.610273    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnpcf\" (UniqueName: \"kubernetes.io/projected/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-kube-api-access-xnpcf\") pod \"payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh\" (UID: \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\") " pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:41.610423 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.610349    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-csdgm\" (UniqueName: \"kubernetes.io/projected/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-kube-api-access-csdgm\") pod \"payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl\" (UID: \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:41.676454 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.676423    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:41.706938 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.706911    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:41.714055 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.714026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:43:41.832799 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.832768    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"]
Jul 25 01:43:41.835795 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:43:41.835583    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec2d5090_b4b1_4a7a_b9d7_d6856fe71f24.slice/crio-0b64d3ef70559a9947a37b5a35844c0b379ec473b4b28058dd0d0bf9ff2d2a0d WatchSource:0}: Error finding container 0b64d3ef70559a9947a37b5a35844c0b379ec473b4b28058dd0d0bf9ff2d2a0d: Status 404 returned error can't find the container with id 0b64d3ef70559a9947a37b5a35844c0b379ec473b4b28058dd0d0bf9ff2d2a0d
Jul 25 01:43:41.861789 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:41.861767    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"]
Jul 25 01:43:41.873079 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:43:41.873052    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0f8a6b_95c4_4dc5_9a7e_7ada293ec4dc.slice/crio-b957fe69e447b36949061511031beb92871fea9b147affde749b2d71b0f34d8d WatchSource:0}: Error finding container b957fe69e447b36949061511031beb92871fea9b147affde749b2d71b0f34d8d: Status 404 returned error can't find the container with id b957fe69e447b36949061511031beb92871fea9b147affde749b2d71b0f34d8d
Jul 25 01:43:42.799962 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:42.799927    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh" event={"ID":"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc","Type":"ContainerStarted","Data":"1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c"}
Jul 25 01:43:42.800451 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:42.800012    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh" event={"ID":"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc","Type":"ContainerStarted","Data":"b957fe69e447b36949061511031beb92871fea9b147affde749b2d71b0f34d8d"}
Jul 25 01:43:42.800451 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:42.800036    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:43:42.801547 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:42.801528    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl" event={"ID":"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24","Type":"ContainerStarted","Data":"bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300"}
Jul 25 01:43:42.801644 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:42.801564    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl" event={"ID":"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24","Type":"ContainerStarted","Data":"0b64d3ef70559a9947a37b5a35844c0b379ec473b4b28058dd0d0bf9ff2d2a0d"}
Jul 25 01:43:42.801644 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:42.801584    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:42.816202 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:42.816154    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh" podStartSLOduration=1.816141937 podStartE2EDuration="1.816141937s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:43:42.814651718 +0000 UTC m=+1901.285608077" watchObservedRunningTime="2026-07-25 01:43:42.816141937 +0000 UTC m=+1901.287098274"
Jul 25 01:43:42.828933 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:42.828896    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl" podStartSLOduration=1.828883005 podStartE2EDuration="1.828883005s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:43:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:43:42.827778053 +0000 UTC m=+1901.298734392" watchObservedRunningTime="2026-07-25 01:43:42.828883005 +0000 UTC m=+1901.299839344"
Jul 25 01:43:53.809069 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:53.809042    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:43:53.809440 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:43:53.809107    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:44:16.115708 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.115675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:16.149871 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.149824    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"]
Jul 25 01:44:16.150159 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.150101    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh" podUID="6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc" containerName="payload-processing" containerID="cri-o://1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c" gracePeriod=30
Jul 25 01:44:16.152818 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.152788    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"]
Jul 25 01:44:16.153035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.153009    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl" podUID="ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24" containerName="payload-pre-processing" containerID="cri-o://bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300" gracePeriod=30
Jul 25 01:44:16.187219 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.187195    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:16.248097 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.248071    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:16.420934 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.420910    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:44:16.444138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.444116    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:44:16.488192 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.488169    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csdgm\" (UniqueName: \"kubernetes.io/projected/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-kube-api-access-csdgm\") pod \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\" (UID: \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\") "
Jul 25 01:44:16.488300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.488242    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnpcf\" (UniqueName: \"kubernetes.io/projected/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-kube-api-access-xnpcf\") pod \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\" (UID: \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\") "
Jul 25 01:44:16.488300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.488271    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-plugins-config-volume\") pod \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\" (UID: \"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc\") "
Jul 25 01:44:16.488300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.488294    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-plugins-config-volume\") pod \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\" (UID: \"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24\") "
Jul 25 01:44:16.488638 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.488611    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc" (UID: "6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:44:16.488748 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.488677    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24" (UID: "ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:44:16.490214 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.490194    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-kube-api-access-csdgm" (OuterVolumeSpecName: "kube-api-access-csdgm") pod "ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24" (UID: "ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24"). InnerVolumeSpecName "kube-api-access-csdgm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:44:16.490348 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.490328    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-kube-api-access-xnpcf" (OuterVolumeSpecName: "kube-api-access-xnpcf") pod "6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc" (UID: "6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc"). InnerVolumeSpecName "kube-api-access-xnpcf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:44:16.589117 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.589095    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:44:16.589117 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.589114    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csdgm\" (UniqueName: \"kubernetes.io/projected/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24-kube-api-access-csdgm\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:44:16.589272 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.589126    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnpcf\" (UniqueName: \"kubernetes.io/projected/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-kube-api-access-xnpcf\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:44:16.589272 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.589134    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:44:16.964713 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.964678    2562 generic.go:356] "Generic (PLEG): container finished" podID="6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc" containerID="1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c" exitCode=0
Jul 25 01:44:16.964887 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.964729    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"
Jul 25 01:44:16.964887 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.964760    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh" event={"ID":"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc","Type":"ContainerDied","Data":"1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c"}
Jul 25 01:44:16.964887 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.964852    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh" event={"ID":"6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc","Type":"ContainerDied","Data":"b957fe69e447b36949061511031beb92871fea9b147affde749b2d71b0f34d8d"}
Jul 25 01:44:16.965106 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.964886    2562 scope.go:117] "RemoveContainer" containerID="1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c"
Jul 25 01:44:16.966183 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.966156    2562 generic.go:356] "Generic (PLEG): container finished" podID="ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24" containerID="bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300" exitCode=0
Jul 25 01:44:16.966263 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.966184    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl" event={"ID":"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24","Type":"ContainerDied","Data":"bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300"}
Jul 25 01:44:16.966263 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.966212    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"
Jul 25 01:44:16.966263 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.966231    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl" event={"ID":"ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24","Type":"ContainerDied","Data":"0b64d3ef70559a9947a37b5a35844c0b379ec473b4b28058dd0d0bf9ff2d2a0d"}
Jul 25 01:44:16.974719 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.974698    2562 scope.go:117] "RemoveContainer" containerID="1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c"
Jul 25 01:44:16.975037 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:44:16.974966    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c\": container with ID starting with 1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c not found: ID does not exist" containerID="1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c"
Jul 25 01:44:16.975133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.975043    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c"} err="failed to get container status \"1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c\": rpc error: code = NotFound desc = could not find container \"1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c\": container with ID starting with 1d2bffa0e6ede712c30f84f5a7583681e73331e8a41b9bd1de8903b341c2df2c not found: ID does not exist"
Jul 25 01:44:16.975133 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.975060    2562 scope.go:117] "RemoveContainer" containerID="bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300"
Jul 25 01:44:16.982726 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.982710    2562 scope.go:117] "RemoveContainer" containerID="bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300"
Jul 25 01:44:16.983032 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:44:16.982960    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300\": container with ID starting with bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300 not found: ID does not exist" containerID="bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300"
Jul 25 01:44:16.983032 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.982999    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300"} err="failed to get container status \"bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300\": rpc error: code = NotFound desc = could not find container \"bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300\": container with ID starting with bc7edb7de806292f95f5c83f0372502ee423f736c1616cb962fa938321f33300 not found: ID does not exist"
Jul 25 01:44:16.987134 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.987112    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"]
Jul 25 01:44:16.988949 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.988931    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-906a2829-86cb56dd58-vsqsh"]
Jul 25 01:44:16.998608 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:16.998583    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"]
Jul 25 01:44:17.000504 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:17.000488    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-906a2829-79988bcb76-jk9cl"]
Jul 25 01:44:18.075634 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:18.075605    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc" path="/var/lib/kubelet/pods/6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc/volumes"
Jul 25 01:44:18.076056 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:18.075968    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24" path="/var/lib/kubelet/pods/ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24/volumes"
Jul 25 01:44:20.674932 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:20.674900    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:20.842458 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:20.842423    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:21.078011 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:21.077924    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:21.574492 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:21.574465    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:21.671235 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:21.671201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:21.772201 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:21.772172    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:23.326361 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:23.326328    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:29.012597 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:29.012562    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:29.114898 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:29.114860    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:29.212825 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:29.212772    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:29.315257 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:29.315153    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:29.754357 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:29.754323    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:52.404804 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:52.404751    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:52.454376 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:52.454347    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:44:55.734138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:44:55.734063    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:01.467536 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:01.467500    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:01.575437 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:01.575381    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:01.793010 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:01.792906    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:17.640256 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:17.640219    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:23.423292 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:23.423243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:23.526841 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:23.526809    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:23.631254 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:23.631215    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:34.183422 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:34.183389    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:35.017963 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:35.017931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:41.071172 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:41.071138    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:41.176168 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:41.176136    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:41.274626 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:41.274589    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:41.374604 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:41.374582    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:55.092119 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:55.092077    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:56.512135 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:56.512104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:45:57.090259 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:45:57.090226    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:02.802130 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:02.802096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:02.903358 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:02.903325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:03.483814 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:03.483773    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:09.272227 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:09.272188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:09.330134 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:09.330104    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:09.512320 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:09.512287    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:09.819833 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:09.819799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:10.157151 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:10.157122    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:13.073446 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:13.073373    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:13.174686 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:13.174640    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:13.426840 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:13.426813    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:13.559453 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:13.559408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:13.997019 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:13.996964    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:14.092851 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.092799    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:14.096837 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.096809    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"]
Jul 25 01:46:14.097207 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.097193    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24" containerName="payload-pre-processing"
Jul 25 01:46:14.097266 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.097208    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24" containerName="payload-pre-processing"
Jul 25 01:46:14.097266 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.097218    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc" containerName="payload-processing"
Jul 25 01:46:14.097266 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.097223    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc" containerName="payload-processing"
Jul 25 01:46:14.097360 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.097290    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d0f8a6b-95c4-4dc5-9a7e-7ada293ec4dc" containerName="payload-processing"
Jul 25 01:46:14.097360 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.097303    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ec2d5090-b4b1-4a7a-b9d7-d6856fe71f24" containerName="payload-pre-processing"
Jul 25 01:46:14.100480 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.100464    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:14.103957 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.103935    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-6d92f36d-dockercfg-cskrj\""
Jul 25 01:46:14.104082 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.103945    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-6d92f36d\""
Jul 25 01:46:14.113460 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.113434    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"]
Jul 25 01:46:14.132658 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.132631    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"]
Jul 25 01:46:14.136168 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.136150    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:14.143840 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.143811    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"]
Jul 25 01:46:14.196906 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.196874    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-schlz\" (UniqueName: \"kubernetes.io/projected/32d5a299-5e27-4e82-9744-2c6e73353de8-kube-api-access-schlz\") pod \"payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6\" (UID: \"32d5a299-5e27-4e82-9744-2c6e73353de8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:14.197128 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.196952    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32d5a299-5e27-4e82-9744-2c6e73353de8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6\" (UID: \"32d5a299-5e27-4e82-9744-2c6e73353de8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:14.298026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.297930    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-schlz\" (UniqueName: \"kubernetes.io/projected/32d5a299-5e27-4e82-9744-2c6e73353de8-kube-api-access-schlz\") pod \"payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6\" (UID: \"32d5a299-5e27-4e82-9744-2c6e73353de8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:14.298026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.297965    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58b05a9d-5429-4918-8d27-db64d0b3c611-plugins-config-volume\") pod \"payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf\" (UID: \"58b05a9d-5429-4918-8d27-db64d0b3c611\") " pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:14.298228 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.298067    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32d5a299-5e27-4e82-9744-2c6e73353de8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6\" (UID: \"32d5a299-5e27-4e82-9744-2c6e73353de8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:14.298228 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.298093    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sx69c\" (UniqueName: \"kubernetes.io/projected/58b05a9d-5429-4918-8d27-db64d0b3c611-kube-api-access-sx69c\") pod \"payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf\" (UID: \"58b05a9d-5429-4918-8d27-db64d0b3c611\") " pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:14.298735 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.298718    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32d5a299-5e27-4e82-9744-2c6e73353de8-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6\" (UID: \"32d5a299-5e27-4e82-9744-2c6e73353de8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:14.305509 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.305487    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-schlz\" (UniqueName: \"kubernetes.io/projected/32d5a299-5e27-4e82-9744-2c6e73353de8-kube-api-access-schlz\") pod \"payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6\" (UID: \"32d5a299-5e27-4e82-9744-2c6e73353de8\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:14.399321 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.399287    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58b05a9d-5429-4918-8d27-db64d0b3c611-plugins-config-volume\") pod \"payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf\" (UID: \"58b05a9d-5429-4918-8d27-db64d0b3c611\") " pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:14.399485 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.399364    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sx69c\" (UniqueName: \"kubernetes.io/projected/58b05a9d-5429-4918-8d27-db64d0b3c611-kube-api-access-sx69c\") pod \"payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf\" (UID: \"58b05a9d-5429-4918-8d27-db64d0b3c611\") " pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:14.400041 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.400018    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58b05a9d-5429-4918-8d27-db64d0b3c611-plugins-config-volume\") pod \"payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf\" (UID: \"58b05a9d-5429-4918-8d27-db64d0b3c611\") " pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:14.407677 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.407655    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sx69c\" (UniqueName: \"kubernetes.io/projected/58b05a9d-5429-4918-8d27-db64d0b3c611-kube-api-access-sx69c\") pod \"payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf\" (UID: \"58b05a9d-5429-4918-8d27-db64d0b3c611\") " pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:14.410424 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.410406    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:14.449698 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.449669    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:14.762800 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.762773    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"]
Jul 25 01:46:14.764548 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:46:14.764514    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32d5a299_5e27_4e82_9744_2c6e73353de8.slice/crio-9229be11a3dfadc015a68eea27861b03863772a2a3a612a59f9e09b00967597f WatchSource:0}: Error finding container 9229be11a3dfadc015a68eea27861b03863772a2a3a612a59f9e09b00967597f: Status 404 returned error can't find the container with id 9229be11a3dfadc015a68eea27861b03863772a2a3a612a59f9e09b00967597f
Jul 25 01:46:14.805018 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:14.804995    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"]
Jul 25 01:46:14.806755 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:46:14.806734    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58b05a9d_5429_4918_8d27_db64d0b3c611.slice/crio-956b47e02db2697fd788c2c51439ff9bb2bc944fa458f957a6bca91b961be919 WatchSource:0}: Error finding container 956b47e02db2697fd788c2c51439ff9bb2bc944fa458f957a6bca91b961be919: Status 404 returned error can't find the container with id 956b47e02db2697fd788c2c51439ff9bb2bc944fa458f957a6bca91b961be919
Jul 25 01:46:15.538141 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:15.537955    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6" event={"ID":"32d5a299-5e27-4e82-9744-2c6e73353de8","Type":"ContainerStarted","Data":"8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c"}
Jul 25 01:46:15.538141 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:15.538039    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6" event={"ID":"32d5a299-5e27-4e82-9744-2c6e73353de8","Type":"ContainerStarted","Data":"9229be11a3dfadc015a68eea27861b03863772a2a3a612a59f9e09b00967597f"}
Jul 25 01:46:15.538687 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:15.538154    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:15.539843 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:15.539821    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf" event={"ID":"58b05a9d-5429-4918-8d27-db64d0b3c611","Type":"ContainerStarted","Data":"114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1"}
Jul 25 01:46:15.540017 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:15.539863    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf" event={"ID":"58b05a9d-5429-4918-8d27-db64d0b3c611","Type":"ContainerStarted","Data":"956b47e02db2697fd788c2c51439ff9bb2bc944fa458f957a6bca91b961be919"}
Jul 25 01:46:15.540017 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:15.539892    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:15.554920 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:15.554875    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6" podStartSLOduration=1.554857121 podStartE2EDuration="1.554857121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:46:15.554778058 +0000 UTC m=+2054.025734397" watchObservedRunningTime="2026-07-25 01:46:15.554857121 +0000 UTC m=+2054.025813462"
Jul 25 01:46:15.571100 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:15.571061    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf" podStartSLOduration=1.571050989 podStartE2EDuration="1.571050989s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:46:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:46:15.570250014 +0000 UTC m=+2054.041206384" watchObservedRunningTime="2026-07-25 01:46:15.571050989 +0000 UTC m=+2054.042007350"
Jul 25 01:46:17.617911 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:17.617867    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:18.326420 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:18.326365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:18.425058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:18.425021    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:18.785701 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:18.785672    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:24.561431 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:24.561347    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:24.664744 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:24.664712    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:26.305686 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:26.305630    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:26.548071 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:26.548030    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6" podUID="32d5a299-5e27-4e82-9744-2c6e73353de8" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.54:9004: connect: connection refused"
Jul 25 01:46:26.548680 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:26.548658    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:35.144362 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.144325    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:35.171991 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.171946    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"]
Jul 25 01:46:35.172266 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.172238    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf" podUID="58b05a9d-5429-4918-8d27-db64d0b3c611" containerName="payload-processing" containerID="cri-o://114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1" gracePeriod=30
Jul 25 01:46:35.177380 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.177354    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"]
Jul 25 01:46:35.177626 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.177588    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6" podUID="32d5a299-5e27-4e82-9744-2c6e73353de8" containerName="payload-pre-processing" containerID="cri-o://8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c" gracePeriod=30
Jul 25 01:46:35.178364 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.178307    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:35.220721 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.220660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:35.276469 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.276444    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:35.454294 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.454268    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:35.457097 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.457078    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:35.487184 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.487157    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58b05a9d-5429-4918-8d27-db64d0b3c611-plugins-config-volume\") pod \"58b05a9d-5429-4918-8d27-db64d0b3c611\" (UID: \"58b05a9d-5429-4918-8d27-db64d0b3c611\") "
Jul 25 01:46:35.487305 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.487237    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sx69c\" (UniqueName: \"kubernetes.io/projected/58b05a9d-5429-4918-8d27-db64d0b3c611-kube-api-access-sx69c\") pod \"58b05a9d-5429-4918-8d27-db64d0b3c611\" (UID: \"58b05a9d-5429-4918-8d27-db64d0b3c611\") "
Jul 25 01:46:35.487367 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.487311    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32d5a299-5e27-4e82-9744-2c6e73353de8-plugins-config-volume\") pod \"32d5a299-5e27-4e82-9744-2c6e73353de8\" (UID: \"32d5a299-5e27-4e82-9744-2c6e73353de8\") "
Jul 25 01:46:35.487367 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.487355    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-schlz\" (UniqueName: \"kubernetes.io/projected/32d5a299-5e27-4e82-9744-2c6e73353de8-kube-api-access-schlz\") pod \"32d5a299-5e27-4e82-9744-2c6e73353de8\" (UID: \"32d5a299-5e27-4e82-9744-2c6e73353de8\") "
Jul 25 01:46:35.487529 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.487500    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58b05a9d-5429-4918-8d27-db64d0b3c611-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58b05a9d-5429-4918-8d27-db64d0b3c611" (UID: "58b05a9d-5429-4918-8d27-db64d0b3c611"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:46:35.487679 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.487660    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/32d5a299-5e27-4e82-9744-2c6e73353de8-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "32d5a299-5e27-4e82-9744-2c6e73353de8" (UID: "32d5a299-5e27-4e82-9744-2c6e73353de8"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:46:35.489418 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.489397    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58b05a9d-5429-4918-8d27-db64d0b3c611-kube-api-access-sx69c" (OuterVolumeSpecName: "kube-api-access-sx69c") pod "58b05a9d-5429-4918-8d27-db64d0b3c611" (UID: "58b05a9d-5429-4918-8d27-db64d0b3c611"). InnerVolumeSpecName "kube-api-access-sx69c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:46:35.489616 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.489600    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32d5a299-5e27-4e82-9744-2c6e73353de8-kube-api-access-schlz" (OuterVolumeSpecName: "kube-api-access-schlz") pod "32d5a299-5e27-4e82-9744-2c6e73353de8" (UID: "32d5a299-5e27-4e82-9744-2c6e73353de8"). InnerVolumeSpecName "kube-api-access-schlz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:46:35.588861 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.588837    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sx69c\" (UniqueName: \"kubernetes.io/projected/58b05a9d-5429-4918-8d27-db64d0b3c611-kube-api-access-sx69c\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:46:35.588861 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.588859    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/32d5a299-5e27-4e82-9744-2c6e73353de8-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:46:35.589008 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.588868    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-schlz\" (UniqueName: \"kubernetes.io/projected/32d5a299-5e27-4e82-9744-2c6e73353de8-kube-api-access-schlz\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:46:35.589008 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.588877    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58b05a9d-5429-4918-8d27-db64d0b3c611-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:46:35.643125 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.643095    2562 generic.go:356] "Generic (PLEG): container finished" podID="58b05a9d-5429-4918-8d27-db64d0b3c611" containerID="114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1" exitCode=0
Jul 25 01:46:35.643238 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.643130    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"
Jul 25 01:46:35.643238 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.643175    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf" event={"ID":"58b05a9d-5429-4918-8d27-db64d0b3c611","Type":"ContainerDied","Data":"114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1"}
Jul 25 01:46:35.643368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.643263    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf" event={"ID":"58b05a9d-5429-4918-8d27-db64d0b3c611","Type":"ContainerDied","Data":"956b47e02db2697fd788c2c51439ff9bb2bc944fa458f957a6bca91b961be919"}
Jul 25 01:46:35.643368 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.643299    2562 scope.go:117] "RemoveContainer" containerID="114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1"
Jul 25 01:46:35.644636 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.644593    2562 generic.go:356] "Generic (PLEG): container finished" podID="32d5a299-5e27-4e82-9744-2c6e73353de8" containerID="8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c" exitCode=0
Jul 25 01:46:35.644732 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.644641    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"
Jul 25 01:46:35.644732 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.644717    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6" event={"ID":"32d5a299-5e27-4e82-9744-2c6e73353de8","Type":"ContainerDied","Data":"8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c"}
Jul 25 01:46:35.644838 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.644772    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6" event={"ID":"32d5a299-5e27-4e82-9744-2c6e73353de8","Type":"ContainerDied","Data":"9229be11a3dfadc015a68eea27861b03863772a2a3a612a59f9e09b00967597f"}
Jul 25 01:46:35.653096 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.652964    2562 scope.go:117] "RemoveContainer" containerID="114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1"
Jul 25 01:46:35.653355 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:46:35.653327    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1\": container with ID starting with 114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1 not found: ID does not exist" containerID="114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1"
Jul 25 01:46:35.653430 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.653366    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1"} err="failed to get container status \"114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1\": rpc error: code = NotFound desc = could not find container \"114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1\": container with ID starting with 114c4e448441dfbb910723693e52d47a020c4ea78218d062b9c2201753cdb1c1 not found: ID does not exist"
Jul 25 01:46:35.653430 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.653388    2562 scope.go:117] "RemoveContainer" containerID="8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c"
Jul 25 01:46:35.660938 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.660913    2562 scope.go:117] "RemoveContainer" containerID="8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c"
Jul 25 01:46:35.661147 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:46:35.661128    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c\": container with ID starting with 8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c not found: ID does not exist" containerID="8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c"
Jul 25 01:46:35.661205 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.661152    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c"} err="failed to get container status \"8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c\": rpc error: code = NotFound desc = could not find container \"8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c\": container with ID starting with 8b4edb0f5408b942ba3cc4bf0d336f737a140f10a4eb66a4110789f27a812b9c not found: ID does not exist"
Jul 25 01:46:35.669507 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.669485    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"]
Jul 25 01:46:35.673973 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.673955    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-6d92f36d-59688cd976-5kbh6"]
Jul 25 01:46:35.685681 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.685631    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"]
Jul 25 01:46:35.689699 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:35.689681    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-6d92f36d-7884c8cc78-f6zcf"]
Jul 25 01:46:36.076479 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:36.076403    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32d5a299-5e27-4e82-9744-2c6e73353de8" path="/var/lib/kubelet/pods/32d5a299-5e27-4e82-9744-2c6e73353de8/volumes"
Jul 25 01:46:36.076812 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:36.076798    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58b05a9d-5429-4918-8d27-db64d0b3c611" path="/var/lib/kubelet/pods/58b05a9d-5429-4918-8d27-db64d0b3c611/volumes"
Jul 25 01:46:40.037726 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:40.037696    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:40.427021 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:40.426965    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:46.195958 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:46.195929    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:46.296674 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:46.296642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:46.479537 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:46.479458    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:46.717489 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:46.717451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:46.835494 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:46.835421    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:46.933587 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:46.933541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:47.064577 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:47.064546    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:52.806025 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:52.805972    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:52.880726 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:52.880690    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:52.964820 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:52.964784    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:53.063481 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:53.063408    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:53.176357 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:53.176333    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:58.723700 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:58.723670    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:46:58.785732 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:46:58.785705    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:05.812902 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:05.812868    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:06.163494 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:06.163463    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:06.501062 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:06.500969    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:09.384058 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:09.384027    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:09.484613 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:09.484574    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:09.609206 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:09.609175    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:10.087472 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.087434    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:10.189538 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.189463    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:10.275170 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.275139    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"]
Jul 25 01:47:10.275629 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.275611    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32d5a299-5e27-4e82-9744-2c6e73353de8" containerName="payload-pre-processing"
Jul 25 01:47:10.275629 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.275629    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="32d5a299-5e27-4e82-9744-2c6e73353de8" containerName="payload-pre-processing"
Jul 25 01:47:10.275827 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.275640    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58b05a9d-5429-4918-8d27-db64d0b3c611" containerName="payload-processing"
Jul 25 01:47:10.275827 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.275646    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="58b05a9d-5429-4918-8d27-db64d0b3c611" containerName="payload-processing"
Jul 25 01:47:10.275827 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.275722    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:10.275827 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.275741    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="32d5a299-5e27-4e82-9744-2c6e73353de8" containerName="payload-pre-processing"
Jul 25 01:47:10.275827 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.275754    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="58b05a9d-5429-4918-8d27-db64d0b3c611" containerName="payload-processing"
Jul 25 01:47:10.278720 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.278705    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:10.281651 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.281633    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-5e11b867\""
Jul 25 01:47:10.281763 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.281637    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-5e11b867-dockercfg-vqd2n\""
Jul 25 01:47:10.293835 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.293811    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"]
Jul 25 01:47:10.302880 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.302859    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"]
Jul 25 01:47:10.306583 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.306567    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:10.323325 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.323305    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"]
Jul 25 01:47:10.373057 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.373024    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtvkw\" (UniqueName: \"kubernetes.io/projected/bfef2edb-f745-42ce-8ee1-7afd0f76d561-kube-api-access-wtvkw\") pod \"payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv\" (UID: \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:10.373219 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.373074    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfef2edb-f745-42ce-8ee1-7afd0f76d561-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv\" (UID: \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:10.373219 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.373169    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5rxh\" (UniqueName: \"kubernetes.io/projected/6ae6898c-2f0d-4dd1-8614-2007bbebca44-kube-api-access-d5rxh\") pod \"payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm\" (UID: \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\") " pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:10.373424 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.373393    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae6898c-2f0d-4dd1-8614-2007bbebca44-plugins-config-volume\") pod \"payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm\" (UID: \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\") " pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:10.475023 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.474965    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5rxh\" (UniqueName: \"kubernetes.io/projected/6ae6898c-2f0d-4dd1-8614-2007bbebca44-kube-api-access-d5rxh\") pod \"payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm\" (UID: \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\") " pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:10.475406 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.475064    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae6898c-2f0d-4dd1-8614-2007bbebca44-plugins-config-volume\") pod \"payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm\" (UID: \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\") " pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:10.475406 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.475102    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtvkw\" (UniqueName: \"kubernetes.io/projected/bfef2edb-f745-42ce-8ee1-7afd0f76d561-kube-api-access-wtvkw\") pod \"payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv\" (UID: \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:10.475406 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.475119    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfef2edb-f745-42ce-8ee1-7afd0f76d561-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv\" (UID: \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:10.475772 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.475754    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfef2edb-f745-42ce-8ee1-7afd0f76d561-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv\" (UID: \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:10.475818 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.475756    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae6898c-2f0d-4dd1-8614-2007bbebca44-plugins-config-volume\") pod \"payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm\" (UID: \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\") " pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:10.483467 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.483443    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5rxh\" (UniqueName: \"kubernetes.io/projected/6ae6898c-2f0d-4dd1-8614-2007bbebca44-kube-api-access-d5rxh\") pod \"payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm\" (UID: \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\") " pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:10.483634 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.483616    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtvkw\" (UniqueName: \"kubernetes.io/projected/bfef2edb-f745-42ce-8ee1-7afd0f76d561-kube-api-access-wtvkw\") pod \"payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv\" (UID: \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:10.590459 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.590418    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:10.616778 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.616754    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:10.771232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.771208    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"]
Jul 25 01:47:10.773466 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:47:10.773440    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae6898c_2f0d_4dd1_8614_2007bbebca44.slice/crio-17fbd1e35f25796e63903a689bf44d05d9fe2b8315eef3b01c1c466b2429b0d6 WatchSource:0}: Error finding container 17fbd1e35f25796e63903a689bf44d05d9fe2b8315eef3b01c1c466b2429b0d6: Status 404 returned error can't find the container with id 17fbd1e35f25796e63903a689bf44d05d9fe2b8315eef3b01c1c466b2429b0d6
Jul 25 01:47:10.814216 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.814185    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm" event={"ID":"6ae6898c-2f0d-4dd1-8614-2007bbebca44","Type":"ContainerStarted","Data":"17fbd1e35f25796e63903a689bf44d05d9fe2b8315eef3b01c1c466b2429b0d6"}
Jul 25 01:47:10.943964 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:10.943921    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"]
Jul 25 01:47:10.945695 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:47:10.945664    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfef2edb_f745_42ce_8ee1_7afd0f76d561.slice/crio-8cd01bf2ab640a252fa215da597becf1ec11e514c037365957c70aed78028643 WatchSource:0}: Error finding container 8cd01bf2ab640a252fa215da597becf1ec11e514c037365957c70aed78028643: Status 404 returned error can't find the container with id 8cd01bf2ab640a252fa215da597becf1ec11e514c037365957c70aed78028643
Jul 25 01:47:11.820632 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:11.820587    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm" event={"ID":"6ae6898c-2f0d-4dd1-8614-2007bbebca44","Type":"ContainerStarted","Data":"5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29"}
Jul 25 01:47:11.821148 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:11.820708    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:47:11.822112 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:11.822091    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv" event={"ID":"bfef2edb-f745-42ce-8ee1-7afd0f76d561","Type":"ContainerStarted","Data":"9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559"}
Jul 25 01:47:11.822192 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:11.822129    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv" event={"ID":"bfef2edb-f745-42ce-8ee1-7afd0f76d561","Type":"ContainerStarted","Data":"8cd01bf2ab640a252fa215da597becf1ec11e514c037365957c70aed78028643"}
Jul 25 01:47:11.822192 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:11.822149    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:11.847508 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:11.847453    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm" podStartSLOduration=1.8474353159999999 podStartE2EDuration="1.847435316s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:47:11.846535978 +0000 UTC m=+2110.317492319" watchObservedRunningTime="2026-07-25 01:47:11.847435316 +0000 UTC m=+2110.318391659"
Jul 25 01:47:11.883786 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:11.883739    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv" podStartSLOduration=1.883722248 podStartE2EDuration="1.883722248s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:47:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:47:11.882738887 +0000 UTC m=+2110.353695224" watchObservedRunningTime="2026-07-25 01:47:11.883722248 +0000 UTC m=+2110.354678588"
Jul 25 01:47:13.930508 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:13.930477    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:22.019017 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:22.018958    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:47:22.829732 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:22.829700    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:47:22.829894 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:47:22.829786    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:48:14.460874 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.460788    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:48:14.486382 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.486338    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"]
Jul 25 01:48:14.486596 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.486542    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm" podUID="6ae6898c-2f0d-4dd1-8614-2007bbebca44" containerName="payload-processing" containerID="cri-o://5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29" gracePeriod=30
Jul 25 01:48:14.494180 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.493893    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"]
Jul 25 01:48:14.496135 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.494407    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv" podUID="bfef2edb-f745-42ce-8ee1-7afd0f76d561" containerName="payload-pre-processing" containerID="cri-o://9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559" gracePeriod=30
Jul 25 01:48:14.563229 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.563201    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:48:14.772772 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.772751    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:48:14.775696 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.775678    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:48:14.913541 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.913513    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfef2edb-f745-42ce-8ee1-7afd0f76d561-plugins-config-volume\") pod \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\" (UID: \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\") "
Jul 25 01:48:14.913693 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.913601    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5rxh\" (UniqueName: \"kubernetes.io/projected/6ae6898c-2f0d-4dd1-8614-2007bbebca44-kube-api-access-d5rxh\") pod \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\" (UID: \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\") "
Jul 25 01:48:14.913693 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.913637    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae6898c-2f0d-4dd1-8614-2007bbebca44-plugins-config-volume\") pod \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\" (UID: \"6ae6898c-2f0d-4dd1-8614-2007bbebca44\") "
Jul 25 01:48:14.913693 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.913681    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtvkw\" (UniqueName: \"kubernetes.io/projected/bfef2edb-f745-42ce-8ee1-7afd0f76d561-kube-api-access-wtvkw\") pod \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\" (UID: \"bfef2edb-f745-42ce-8ee1-7afd0f76d561\") "
Jul 25 01:48:14.913898 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.913878    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfef2edb-f745-42ce-8ee1-7afd0f76d561-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "bfef2edb-f745-42ce-8ee1-7afd0f76d561" (UID: "bfef2edb-f745-42ce-8ee1-7afd0f76d561"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:48:14.914025 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.914005    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/bfef2edb-f745-42ce-8ee1-7afd0f76d561-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:48:14.914094 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.914068    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae6898c-2f0d-4dd1-8614-2007bbebca44-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6ae6898c-2f0d-4dd1-8614-2007bbebca44" (UID: "6ae6898c-2f0d-4dd1-8614-2007bbebca44"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:48:14.915695 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.915676    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ae6898c-2f0d-4dd1-8614-2007bbebca44-kube-api-access-d5rxh" (OuterVolumeSpecName: "kube-api-access-d5rxh") pod "6ae6898c-2f0d-4dd1-8614-2007bbebca44" (UID: "6ae6898c-2f0d-4dd1-8614-2007bbebca44"). InnerVolumeSpecName "kube-api-access-d5rxh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:48:14.915800 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:14.915784    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfef2edb-f745-42ce-8ee1-7afd0f76d561-kube-api-access-wtvkw" (OuterVolumeSpecName: "kube-api-access-wtvkw") pod "bfef2edb-f745-42ce-8ee1-7afd0f76d561" (UID: "bfef2edb-f745-42ce-8ee1-7afd0f76d561"). InnerVolumeSpecName "kube-api-access-wtvkw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:48:15.014537 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.014484    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5rxh\" (UniqueName: \"kubernetes.io/projected/6ae6898c-2f0d-4dd1-8614-2007bbebca44-kube-api-access-d5rxh\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:48:15.014537 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.014502    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6ae6898c-2f0d-4dd1-8614-2007bbebca44-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:48:15.014537 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.014512    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtvkw\" (UniqueName: \"kubernetes.io/projected/bfef2edb-f745-42ce-8ee1-7afd0f76d561-kube-api-access-wtvkw\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:48:15.150833 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.150805    2562 generic.go:356] "Generic (PLEG): container finished" podID="6ae6898c-2f0d-4dd1-8614-2007bbebca44" containerID="5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29" exitCode=0
Jul 25 01:48:15.151011 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.150848    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"
Jul 25 01:48:15.151011 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.150876    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm" event={"ID":"6ae6898c-2f0d-4dd1-8614-2007bbebca44","Type":"ContainerDied","Data":"5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29"}
Jul 25 01:48:15.151011 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.150928    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm" event={"ID":"6ae6898c-2f0d-4dd1-8614-2007bbebca44","Type":"ContainerDied","Data":"17fbd1e35f25796e63903a689bf44d05d9fe2b8315eef3b01c1c466b2429b0d6"}
Jul 25 01:48:15.151011 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.150954    2562 scope.go:117] "RemoveContainer" containerID="5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29"
Jul 25 01:48:15.152351 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.152324    2562 generic.go:356] "Generic (PLEG): container finished" podID="bfef2edb-f745-42ce-8ee1-7afd0f76d561" containerID="9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559" exitCode=0
Jul 25 01:48:15.152459 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.152378    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv" event={"ID":"bfef2edb-f745-42ce-8ee1-7afd0f76d561","Type":"ContainerDied","Data":"9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559"}
Jul 25 01:48:15.152459 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.152365    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"
Jul 25 01:48:15.152459 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.152423    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv" event={"ID":"bfef2edb-f745-42ce-8ee1-7afd0f76d561","Type":"ContainerDied","Data":"8cd01bf2ab640a252fa215da597becf1ec11e514c037365957c70aed78028643"}
Jul 25 01:48:15.160569 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.160551    2562 scope.go:117] "RemoveContainer" containerID="5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29"
Jul 25 01:48:15.160842 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:48:15.160825    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29\": container with ID starting with 5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29 not found: ID does not exist" containerID="5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29"
Jul 25 01:48:15.160889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.160851    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29"} err="failed to get container status \"5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29\": rpc error: code = NotFound desc = could not find container \"5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29\": container with ID starting with 5da940517365e9993e36893b6254c9abca3ec1d645f328fbfe8e1c3192df9f29 not found: ID does not exist"
Jul 25 01:48:15.160889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.160872    2562 scope.go:117] "RemoveContainer" containerID="9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559"
Jul 25 01:48:15.168588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.168574    2562 scope.go:117] "RemoveContainer" containerID="9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559"
Jul 25 01:48:15.168810 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:48:15.168791    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559\": container with ID starting with 9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559 not found: ID does not exist" containerID="9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559"
Jul 25 01:48:15.168872 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.168821    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559"} err="failed to get container status \"9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559\": rpc error: code = NotFound desc = could not find container \"9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559\": container with ID starting with 9157eeb0d3b9369833b95a91b4152f88d1368b624096a015adf90cc916f7d559 not found: ID does not exist"
Jul 25 01:48:15.176599 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.176580    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"]
Jul 25 01:48:15.181143 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.181125    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-5e11b867-6db759fc48-cqhvv"]
Jul 25 01:48:15.194145 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.194125    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"]
Jul 25 01:48:15.197801 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:15.197784    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-5e11b867-cdf959b54-rmlsm"]
Jul 25 01:48:16.075618 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:16.075585    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae6898c-2f0d-4dd1-8614-2007bbebca44" path="/var/lib/kubelet/pods/6ae6898c-2f0d-4dd1-8614-2007bbebca44/volumes"
Jul 25 01:48:16.076139 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:16.076119    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfef2edb-f745-42ce-8ee1-7afd0f76d561" path="/var/lib/kubelet/pods/bfef2edb-f745-42ce-8ee1-7afd0f76d561/volumes"
Jul 25 01:48:19.084738 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:19.084706    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:48:22.493320 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:22.493286    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:48:27.569315 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:48:27.569279    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:04.319989 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:04.319934    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:04.627017 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:04.626961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:10.136860 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:10.136820    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:15.997118 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:15.997078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:16.094625 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:16.094591    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:16.285045 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:16.284961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:16.525526 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:16.525496    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:17.660145 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:17.660110    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:17.759189 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:17.759157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:17.894358 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:17.894312    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:18.652294 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:18.652252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:18.750559 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:18.750531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:18.853809 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:18.853760    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:20.305569 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:20.305541    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:26.152069 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:26.151995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:26.254803 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:26.254769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:26.853351 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:26.853308    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:32.789835 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:32.789802    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:32.897886 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:32.897855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:32.991999 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:32.991947    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:39.337996 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:39.337955    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:43.560576 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:43.560538    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:49:43.917745 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:49:43.917717    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:04.934001 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:04.933958    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:11.224564 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:11.224517    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:11.324107 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:11.324078    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:11.598001 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:11.597912    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:11.943844 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:11.943810    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:12.430951 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:12.430921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:15.348095 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:15.348063    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:15.424340 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:15.424316    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:15.501427 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:15.501400    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:15.777631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:15.777558    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:15.966946 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:15.966915    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:16.065601 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.065518    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:16.089628 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.089604    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"]
Jul 25 01:50:16.090135 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.090117    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae6898c-2f0d-4dd1-8614-2007bbebca44" containerName="payload-processing"
Jul 25 01:50:16.090135 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.090135    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae6898c-2f0d-4dd1-8614-2007bbebca44" containerName="payload-processing"
Jul 25 01:50:16.090278 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.090159    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfef2edb-f745-42ce-8ee1-7afd0f76d561" containerName="payload-pre-processing"
Jul 25 01:50:16.090278 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.090169    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfef2edb-f745-42ce-8ee1-7afd0f76d561" containerName="payload-pre-processing"
Jul 25 01:50:16.090278 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.090262    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfef2edb-f745-42ce-8ee1-7afd0f76d561" containerName="payload-pre-processing"
Jul 25 01:50:16.090278 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.090275    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae6898c-2f0d-4dd1-8614-2007bbebca44" containerName="payload-processing"
Jul 25 01:50:16.093274 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.093260    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:16.099339 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.099319    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-df5e5d-dockercfg-9fhsq\""
Jul 25 01:50:16.099473 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.099319    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-df5e5d\""
Jul 25 01:50:16.109339 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.109318    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"]
Jul 25 01:50:16.116946 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.116922    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27h8c\" (UniqueName: \"kubernetes.io/projected/cf8aad7c-9d17-4366-b470-ba04368d6721-kube-api-access-27h8c\") pod \"payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh\" (UID: \"cf8aad7c-9d17-4366-b470-ba04368d6721\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:16.117099 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.117061    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf8aad7c-9d17-4366-b470-ba04368d6721-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh\" (UID: \"cf8aad7c-9d17-4366-b470-ba04368d6721\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:16.117712 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.117692    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"]
Jul 25 01:50:16.120872 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.120856    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.131387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.131363    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"]
Jul 25 01:50:16.158656 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.158627    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:16.217588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.217552    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jppx\" (UniqueName: \"kubernetes.io/projected/a7696e25-54f9-4e75-b25e-f99bb134f5ca-kube-api-access-4jppx\") pod \"payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv\" (UID: \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.217745 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.217618    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7696e25-54f9-4e75-b25e-f99bb134f5ca-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv\" (UID: \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.217745 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.217664    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf8aad7c-9d17-4366-b470-ba04368d6721-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh\" (UID: \"cf8aad7c-9d17-4366-b470-ba04368d6721\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:16.217834 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.217754    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27h8c\" (UniqueName: \"kubernetes.io/projected/cf8aad7c-9d17-4366-b470-ba04368d6721-kube-api-access-27h8c\") pod \"payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh\" (UID: \"cf8aad7c-9d17-4366-b470-ba04368d6721\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:16.218316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.218299    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf8aad7c-9d17-4366-b470-ba04368d6721-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh\" (UID: \"cf8aad7c-9d17-4366-b470-ba04368d6721\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:16.227254 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.227224    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27h8c\" (UniqueName: \"kubernetes.io/projected/cf8aad7c-9d17-4366-b470-ba04368d6721-kube-api-access-27h8c\") pod \"payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh\" (UID: \"cf8aad7c-9d17-4366-b470-ba04368d6721\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:16.318930 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.318845    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7696e25-54f9-4e75-b25e-f99bb134f5ca-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv\" (UID: \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.319082 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.318942    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4jppx\" (UniqueName: \"kubernetes.io/projected/a7696e25-54f9-4e75-b25e-f99bb134f5ca-kube-api-access-4jppx\") pod \"payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv\" (UID: \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.319523 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.319506    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7696e25-54f9-4e75-b25e-f99bb134f5ca-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv\" (UID: \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.328493 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.328467    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jppx\" (UniqueName: \"kubernetes.io/projected/a7696e25-54f9-4e75-b25e-f99bb134f5ca-kube-api-access-4jppx\") pod \"payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv\" (UID: \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\") " pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.403709 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.403687    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:16.430913 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.430893    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.582105 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.582074    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"]
Jul 25 01:50:16.584006 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:50:16.583957    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7696e25_54f9_4e75_b25e_f99bb134f5ca.slice/crio-560a57365a329d9d591d86c732a1e25cd47c767ed4843aefdcf2999044876aff WatchSource:0}: Error finding container 560a57365a329d9d591d86c732a1e25cd47c767ed4843aefdcf2999044876aff: Status 404 returned error can't find the container with id 560a57365a329d9d591d86c732a1e25cd47c767ed4843aefdcf2999044876aff
Jul 25 01:50:16.720027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.719966    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv" event={"ID":"a7696e25-54f9-4e75-b25e-f99bb134f5ca","Type":"ContainerStarted","Data":"78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e"}
Jul 25 01:50:16.720176 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.720048    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv" event={"ID":"a7696e25-54f9-4e75-b25e-f99bb134f5ca","Type":"ContainerStarted","Data":"560a57365a329d9d591d86c732a1e25cd47c767ed4843aefdcf2999044876aff"}
Jul 25 01:50:16.720176 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.720152    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:16.741415 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.741355    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv" podStartSLOduration=0.741335079 podStartE2EDuration="741.335079ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:50:16.738374561 +0000 UTC m=+2295.209330899" watchObservedRunningTime="2026-07-25 01:50:16.741335079 +0000 UTC m=+2295.212291421"
Jul 25 01:50:16.790588 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:16.790567    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"]
Jul 25 01:50:16.792091 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:50:16.792071    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf8aad7c_9d17_4366_b470_ba04368d6721.slice/crio-1756eee4db8a65d02bda3861f4a731fae36aa13c2f9dfc7e4227e79d4e5e8e9a WatchSource:0}: Error finding container 1756eee4db8a65d02bda3861f4a731fae36aa13c2f9dfc7e4227e79d4e5e8e9a: Status 404 returned error can't find the container with id 1756eee4db8a65d02bda3861f4a731fae36aa13c2f9dfc7e4227e79d4e5e8e9a
Jul 25 01:50:17.726950 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:17.726913    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh" event={"ID":"cf8aad7c-9d17-4366-b470-ba04368d6721","Type":"ContainerStarted","Data":"e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2"}
Jul 25 01:50:17.727377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:17.727271    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:17.727377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:17.727315    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh" event={"ID":"cf8aad7c-9d17-4366-b470-ba04368d6721","Type":"ContainerStarted","Data":"1756eee4db8a65d02bda3861f4a731fae36aa13c2f9dfc7e4227e79d4e5e8e9a"}
Jul 25 01:50:17.746533 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:17.746486    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh" podStartSLOduration=1.746473397 podStartE2EDuration="1.746473397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:50:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:50:17.743163172 +0000 UTC m=+2296.214119510" watchObservedRunningTime="2026-07-25 01:50:17.746473397 +0000 UTC m=+2296.217429732"
Jul 25 01:50:19.903693 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:19.903660    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:20.248874 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:20.248793    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:20.589281 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:20.589204    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:20.797874 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:20.797846    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:23.399210 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:23.399176    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:23.542693 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:23.542653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:23.677309 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:23.677239    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:24.006085 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.006010    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:24.226811 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.226775    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:24.321049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.320951    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:24.325035 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.325000    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"]
Jul 25 01:50:24.328618 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.328603    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:24.331548 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.331533    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-320e6c-dockercfg-56d7q\""
Jul 25 01:50:24.331624 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.331582    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-320e6c\""
Jul 25 01:50:24.339669 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.339645    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"]
Jul 25 01:50:24.359945 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.359917    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"]
Jul 25 01:50:24.363150 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.363132    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:24.372958 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.372936    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"]
Jul 25 01:50:24.396394 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.396371    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/765cc7b9-0dec-437a-9526-42d297733413-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9\" (UID: \"765cc7b9-0dec-437a-9526-42d297733413\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:24.396506 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.396404    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctpkh\" (UniqueName: \"kubernetes.io/projected/765cc7b9-0dec-437a-9526-42d297733413-kube-api-access-ctpkh\") pod \"payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9\" (UID: \"765cc7b9-0dec-437a-9526-42d297733413\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:24.497849 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.497812    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/765cc7b9-0dec-437a-9526-42d297733413-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9\" (UID: \"765cc7b9-0dec-437a-9526-42d297733413\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:24.497849 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.497849    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctpkh\" (UniqueName: \"kubernetes.io/projected/765cc7b9-0dec-437a-9526-42d297733413-kube-api-access-ctpkh\") pod \"payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9\" (UID: \"765cc7b9-0dec-437a-9526-42d297733413\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:24.498291 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.497896    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f09cd497-de22-42bf-b970-c29c995b35ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths\" (UID: \"f09cd497-de22-42bf-b970-c29c995b35ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:24.498291 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.497924    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hjpn4\" (UniqueName: \"kubernetes.io/projected/f09cd497-de22-42bf-b970-c29c995b35ae-kube-api-access-hjpn4\") pod \"payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths\" (UID: \"f09cd497-de22-42bf-b970-c29c995b35ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:24.498540 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.498522    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/765cc7b9-0dec-437a-9526-42d297733413-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9\" (UID: \"765cc7b9-0dec-437a-9526-42d297733413\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:24.506455 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.506432    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctpkh\" (UniqueName: \"kubernetes.io/projected/765cc7b9-0dec-437a-9526-42d297733413-kube-api-access-ctpkh\") pod \"payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9\" (UID: \"765cc7b9-0dec-437a-9526-42d297733413\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:24.598712 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.598648    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f09cd497-de22-42bf-b970-c29c995b35ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths\" (UID: \"f09cd497-de22-42bf-b970-c29c995b35ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:24.598712 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.598677    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hjpn4\" (UniqueName: \"kubernetes.io/projected/f09cd497-de22-42bf-b970-c29c995b35ae-kube-api-access-hjpn4\") pod \"payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths\" (UID: \"f09cd497-de22-42bf-b970-c29c995b35ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:24.599308 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.599290    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f09cd497-de22-42bf-b970-c29c995b35ae-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths\" (UID: \"f09cd497-de22-42bf-b970-c29c995b35ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:24.612080 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.612060    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hjpn4\" (UniqueName: \"kubernetes.io/projected/f09cd497-de22-42bf-b970-c29c995b35ae-kube-api-access-hjpn4\") pod \"payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths\" (UID: \"f09cd497-de22-42bf-b970-c29c995b35ae\") " pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:24.640993 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.640957    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:24.673122 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.673088    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:24.999337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:24.999311    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"]
Jul 25 01:50:25.001037 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:50:25.001006    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod765cc7b9_0dec_437a_9526_42d297733413.slice/crio-78861e11e9af2d190146453753ee54a914393232e011461bec80f9b515c3e4bf WatchSource:0}: Error finding container 78861e11e9af2d190146453753ee54a914393232e011461bec80f9b515c3e4bf: Status 404 returned error can't find the container with id 78861e11e9af2d190146453753ee54a914393232e011461bec80f9b515c3e4bf
Jul 25 01:50:25.032441 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.032419    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"]
Jul 25 01:50:25.034991 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:50:25.034952    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf09cd497_de22_42bf_b970_c29c995b35ae.slice/crio-8c73e2b386e9fd8ed8c398bfa799647c028360dab98314e627793a23cbdc963a WatchSource:0}: Error finding container 8c73e2b386e9fd8ed8c398bfa799647c028360dab98314e627793a23cbdc963a: Status 404 returned error can't find the container with id 8c73e2b386e9fd8ed8c398bfa799647c028360dab98314e627793a23cbdc963a
Jul 25 01:50:25.771584 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.771551    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths" event={"ID":"f09cd497-de22-42bf-b970-c29c995b35ae","Type":"ContainerStarted","Data":"2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0"}
Jul 25 01:50:25.772082 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.771612    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:25.772082 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.771637    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths" event={"ID":"f09cd497-de22-42bf-b970-c29c995b35ae","Type":"ContainerStarted","Data":"8c73e2b386e9fd8ed8c398bfa799647c028360dab98314e627793a23cbdc963a"}
Jul 25 01:50:25.773052 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.773025    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9" event={"ID":"765cc7b9-0dec-437a-9526-42d297733413","Type":"ContainerStarted","Data":"b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0"}
Jul 25 01:50:25.773167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.773081    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:25.773167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.773104    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9" event={"ID":"765cc7b9-0dec-437a-9526-42d297733413","Type":"ContainerStarted","Data":"78861e11e9af2d190146453753ee54a914393232e011461bec80f9b515c3e4bf"}
Jul 25 01:50:25.794955 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.794908    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths" podStartSLOduration=1.794893828 podStartE2EDuration="1.794893828s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:50:25.791990886 +0000 UTC m=+2304.262947216" watchObservedRunningTime="2026-07-25 01:50:25.794893828 +0000 UTC m=+2304.265850165"
Jul 25 01:50:25.810749 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:25.810706    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9" podStartSLOduration=1.810692929 podStartE2EDuration="1.810692929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:50:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:50:25.808761573 +0000 UTC m=+2304.279717919" watchObservedRunningTime="2026-07-25 01:50:25.810692929 +0000 UTC m=+2304.281649266"
Jul 25 01:50:27.729523 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:27.729497    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:50:27.903028 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:27.902964    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:50:28.733213 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:28.733186    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:50:36.780847 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:36.780820    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:50:36.781464 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:36.780897    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:50:40.726850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:50:40.726815    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:51:19.302250 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:51:19.302156    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:51:28.954917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:51:28.954874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:51:29.055354 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:51:29.055327    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:51:37.916820 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:51:37.916790    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:51:59.037201 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:51:59.037117    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:03.057004 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:03.056931    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:04.046084 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:04.046052    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:14.562957 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.562919    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:14.582272 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.582248    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"]
Jul 25 01:52:14.582503 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.582482    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv" podUID="a7696e25-54f9-4e75-b25e-f99bb134f5ca" containerName="payload-processing" containerID="cri-o://78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e" gracePeriod=30
Jul 25 01:52:14.588212 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.588189    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"]
Jul 25 01:52:14.588704 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.588615    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh" podUID="cf8aad7c-9d17-4366-b470-ba04368d6721" containerName="payload-pre-processing" containerID="cri-o://e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2" gracePeriod=30
Jul 25 01:52:14.662075 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.662048    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:14.716027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.715998    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:14.879227 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.879194    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:52:14.882205 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.882185    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:52:14.947143 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.947108    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf8aad7c-9d17-4366-b470-ba04368d6721-plugins-config-volume\") pod \"cf8aad7c-9d17-4366-b470-ba04368d6721\" (UID: \"cf8aad7c-9d17-4366-b470-ba04368d6721\") "
Jul 25 01:52:14.947311 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.947186    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7696e25-54f9-4e75-b25e-f99bb134f5ca-plugins-config-volume\") pod \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\" (UID: \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\") "
Jul 25 01:52:14.947311 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.947236    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jppx\" (UniqueName: \"kubernetes.io/projected/a7696e25-54f9-4e75-b25e-f99bb134f5ca-kube-api-access-4jppx\") pod \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\" (UID: \"a7696e25-54f9-4e75-b25e-f99bb134f5ca\") "
Jul 25 01:52:14.947311 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.947257    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-27h8c\" (UniqueName: \"kubernetes.io/projected/cf8aad7c-9d17-4366-b470-ba04368d6721-kube-api-access-27h8c\") pod \"cf8aad7c-9d17-4366-b470-ba04368d6721\" (UID: \"cf8aad7c-9d17-4366-b470-ba04368d6721\") "
Jul 25 01:52:14.947557 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.947518    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf8aad7c-9d17-4366-b470-ba04368d6721-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "cf8aad7c-9d17-4366-b470-ba04368d6721" (UID: "cf8aad7c-9d17-4366-b470-ba04368d6721"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:52:14.947557 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.947548    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a7696e25-54f9-4e75-b25e-f99bb134f5ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "a7696e25-54f9-4e75-b25e-f99bb134f5ca" (UID: "a7696e25-54f9-4e75-b25e-f99bb134f5ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:52:14.949293 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.949273    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a7696e25-54f9-4e75-b25e-f99bb134f5ca-kube-api-access-4jppx" (OuterVolumeSpecName: "kube-api-access-4jppx") pod "a7696e25-54f9-4e75-b25e-f99bb134f5ca" (UID: "a7696e25-54f9-4e75-b25e-f99bb134f5ca"). InnerVolumeSpecName "kube-api-access-4jppx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:52:14.949383 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:14.949341    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf8aad7c-9d17-4366-b470-ba04368d6721-kube-api-access-27h8c" (OuterVolumeSpecName: "kube-api-access-27h8c") pod "cf8aad7c-9d17-4366-b470-ba04368d6721" (UID: "cf8aad7c-9d17-4366-b470-ba04368d6721"). InnerVolumeSpecName "kube-api-access-27h8c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:52:15.048055 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.048029    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-27h8c\" (UniqueName: \"kubernetes.io/projected/cf8aad7c-9d17-4366-b470-ba04368d6721-kube-api-access-27h8c\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:52:15.048055 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.048053    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/cf8aad7c-9d17-4366-b470-ba04368d6721-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:52:15.048213 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.048068    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a7696e25-54f9-4e75-b25e-f99bb134f5ca-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:52:15.048213 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.048082    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jppx\" (UniqueName: \"kubernetes.io/projected/a7696e25-54f9-4e75-b25e-f99bb134f5ca-kube-api-access-4jppx\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:52:15.324571 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.324542    2562 generic.go:356] "Generic (PLEG): container finished" podID="a7696e25-54f9-4e75-b25e-f99bb134f5ca" containerID="78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e" exitCode=0
Jul 25 01:52:15.324759 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.324586    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"
Jul 25 01:52:15.324759 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.324628    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv" event={"ID":"a7696e25-54f9-4e75-b25e-f99bb134f5ca","Type":"ContainerDied","Data":"78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e"}
Jul 25 01:52:15.324759 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.324684    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv" event={"ID":"a7696e25-54f9-4e75-b25e-f99bb134f5ca","Type":"ContainerDied","Data":"560a57365a329d9d591d86c732a1e25cd47c767ed4843aefdcf2999044876aff"}
Jul 25 01:52:15.324759 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.324715    2562 scope.go:117] "RemoveContainer" containerID="78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e"
Jul 25 01:52:15.326039 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.326021    2562 generic.go:356] "Generic (PLEG): container finished" podID="cf8aad7c-9d17-4366-b470-ba04368d6721" containerID="e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2" exitCode=0
Jul 25 01:52:15.326140 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.326048    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh" event={"ID":"cf8aad7c-9d17-4366-b470-ba04368d6721","Type":"ContainerDied","Data":"e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2"}
Jul 25 01:52:15.326140 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.326063    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"
Jul 25 01:52:15.326140 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.326078    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh" event={"ID":"cf8aad7c-9d17-4366-b470-ba04368d6721","Type":"ContainerDied","Data":"1756eee4db8a65d02bda3861f4a731fae36aa13c2f9dfc7e4227e79d4e5e8e9a"}
Jul 25 01:52:15.337594 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.337457    2562 scope.go:117] "RemoveContainer" containerID="78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e"
Jul 25 01:52:15.337790 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:52:15.337764    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e\": container with ID starting with 78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e not found: ID does not exist" containerID="78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e"
Jul 25 01:52:15.337864 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.337801    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e"} err="failed to get container status \"78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e\": rpc error: code = NotFound desc = could not find container \"78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e\": container with ID starting with 78f743cd23e927474b1151a21896a3cc4f7cb05e6d129008dbe41eb02904d23e not found: ID does not exist"
Jul 25 01:52:15.337864 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.337826    2562 scope.go:117] "RemoveContainer" containerID="e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2"
Jul 25 01:52:15.346406 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.346389    2562 scope.go:117] "RemoveContainer" containerID="e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2"
Jul 25 01:52:15.346615 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:52:15.346602    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2\": container with ID starting with e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2 not found: ID does not exist" containerID="e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2"
Jul 25 01:52:15.346661 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.346620    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2"} err="failed to get container status \"e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2\": rpc error: code = NotFound desc = could not find container \"e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2\": container with ID starting with e9c19ba120623b0dfca08746375b17828a45d4ab51a4de5ba172ccab920a27f2 not found: ID does not exist"
Jul 25 01:52:15.351688 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.351669    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"]
Jul 25 01:52:15.354877 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.354857    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-df5e5d-5bbdc5cfb-lfdzh"]
Jul 25 01:52:15.364956 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.364938    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"]
Jul 25 01:52:15.368700 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:15.368683    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-df5e5d-5fd4f45cd8-6fddv"]
Jul 25 01:52:16.076543 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:16.076509    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a7696e25-54f9-4e75-b25e-f99bb134f5ca" path="/var/lib/kubelet/pods/a7696e25-54f9-4e75-b25e-f99bb134f5ca/volumes"
Jul 25 01:52:16.076905 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:16.076879    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cf8aad7c-9d17-4366-b470-ba04368d6721" path="/var/lib/kubelet/pods/cf8aad7c-9d17-4366-b470-ba04368d6721/volumes"
Jul 25 01:52:19.245577 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:19.245540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:19.648652 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:19.648618    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:25.455051 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:25.454921    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:25.595378 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:25.595337    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:25.746443 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:25.746356    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:25.981025 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:25.980994    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:37.093155 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.093106    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:37.119003 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.118958    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"]
Jul 25 01:52:37.119187 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.119162    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths" podUID="f09cd497-de22-42bf-b970-c29c995b35ae" containerName="payload-processing" containerID="cri-o://2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0" gracePeriod=30
Jul 25 01:52:37.124821 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.124798    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"]
Jul 25 01:52:37.125050 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.125011    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9" podUID="765cc7b9-0dec-437a-9526-42d297733413" containerName="payload-pre-processing" containerID="cri-o://b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0" gracePeriod=30
Jul 25 01:52:37.161327 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.161256    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:37.213780 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.213754    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:37.416558 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.416535    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:52:37.419350 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.419330    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:52:37.436350 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.436251    2562 generic.go:356] "Generic (PLEG): container finished" podID="f09cd497-de22-42bf-b970-c29c995b35ae" containerID="2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0" exitCode=0
Jul 25 01:52:37.436350 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.436283    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths" event={"ID":"f09cd497-de22-42bf-b970-c29c995b35ae","Type":"ContainerDied","Data":"2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0"}
Jul 25 01:52:37.436350 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.436293    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"
Jul 25 01:52:37.436350 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.436351    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths" event={"ID":"f09cd497-de22-42bf-b970-c29c995b35ae","Type":"ContainerDied","Data":"8c73e2b386e9fd8ed8c398bfa799647c028360dab98314e627793a23cbdc963a"}
Jul 25 01:52:37.436666 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.436386    2562 scope.go:117] "RemoveContainer" containerID="2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0"
Jul 25 01:52:37.438102 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.438078    2562 generic.go:356] "Generic (PLEG): container finished" podID="765cc7b9-0dec-437a-9526-42d297733413" containerID="b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0" exitCode=0
Jul 25 01:52:37.438195 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.438121    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"
Jul 25 01:52:37.438195 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.438151    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9" event={"ID":"765cc7b9-0dec-437a-9526-42d297733413","Type":"ContainerDied","Data":"b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0"}
Jul 25 01:52:37.438326 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.438190    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9" event={"ID":"765cc7b9-0dec-437a-9526-42d297733413","Type":"ContainerDied","Data":"78861e11e9af2d190146453753ee54a914393232e011461bec80f9b515c3e4bf"}
Jul 25 01:52:37.448434 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.448410    2562 scope.go:117] "RemoveContainer" containerID="2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0"
Jul 25 01:52:37.448733 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:52:37.448702    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0\": container with ID starting with 2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0 not found: ID does not exist" containerID="2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0"
Jul 25 01:52:37.448837 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.448744    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0"} err="failed to get container status \"2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0\": rpc error: code = NotFound desc = could not find container \"2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0\": container with ID starting with 2d6a0e02ecb1e595b4586c11dbf8f0de4e5fa8b53e68c60d45d481ba4f8fe6f0 not found: ID does not exist"
Jul 25 01:52:37.448837 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.448766    2562 scope.go:117] "RemoveContainer" containerID="b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0"
Jul 25 01:52:37.457704 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.457687    2562 scope.go:117] "RemoveContainer" containerID="b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0"
Jul 25 01:52:37.457965 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:52:37.457943    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0\": container with ID starting with b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0 not found: ID does not exist" containerID="b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0"
Jul 25 01:52:37.458082 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.457971    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0"} err="failed to get container status \"b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0\": rpc error: code = NotFound desc = could not find container \"b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0\": container with ID starting with b7bfe82c44be919873d5f4c75f047c9e1ca86c52473ad66ce1255917c92eb9d0 not found: ID does not exist"
Jul 25 01:52:37.537101 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.537073    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f09cd497-de22-42bf-b970-c29c995b35ae-plugins-config-volume\") pod \"f09cd497-de22-42bf-b970-c29c995b35ae\" (UID: \"f09cd497-de22-42bf-b970-c29c995b35ae\") "
Jul 25 01:52:37.537274 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.537128    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ctpkh\" (UniqueName: \"kubernetes.io/projected/765cc7b9-0dec-437a-9526-42d297733413-kube-api-access-ctpkh\") pod \"765cc7b9-0dec-437a-9526-42d297733413\" (UID: \"765cc7b9-0dec-437a-9526-42d297733413\") "
Jul 25 01:52:37.537274 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.537159    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/765cc7b9-0dec-437a-9526-42d297733413-plugins-config-volume\") pod \"765cc7b9-0dec-437a-9526-42d297733413\" (UID: \"765cc7b9-0dec-437a-9526-42d297733413\") "
Jul 25 01:52:37.537274 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.537189    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hjpn4\" (UniqueName: \"kubernetes.io/projected/f09cd497-de22-42bf-b970-c29c995b35ae-kube-api-access-hjpn4\") pod \"f09cd497-de22-42bf-b970-c29c995b35ae\" (UID: \"f09cd497-de22-42bf-b970-c29c995b35ae\") "
Jul 25 01:52:37.537565 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.537531    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f09cd497-de22-42bf-b970-c29c995b35ae-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f09cd497-de22-42bf-b970-c29c995b35ae" (UID: "f09cd497-de22-42bf-b970-c29c995b35ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:52:37.537911 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.537885    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/765cc7b9-0dec-437a-9526-42d297733413-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "765cc7b9-0dec-437a-9526-42d297733413" (UID: "765cc7b9-0dec-437a-9526-42d297733413"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:52:37.539421 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.539399    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/765cc7b9-0dec-437a-9526-42d297733413-kube-api-access-ctpkh" (OuterVolumeSpecName: "kube-api-access-ctpkh") pod "765cc7b9-0dec-437a-9526-42d297733413" (UID: "765cc7b9-0dec-437a-9526-42d297733413"). InnerVolumeSpecName "kube-api-access-ctpkh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:52:37.539912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.539887    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f09cd497-de22-42bf-b970-c29c995b35ae-kube-api-access-hjpn4" (OuterVolumeSpecName: "kube-api-access-hjpn4") pod "f09cd497-de22-42bf-b970-c29c995b35ae" (UID: "f09cd497-de22-42bf-b970-c29c995b35ae"). InnerVolumeSpecName "kube-api-access-hjpn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:52:37.638617 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.638553    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f09cd497-de22-42bf-b970-c29c995b35ae-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:52:37.638617 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.638577    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ctpkh\" (UniqueName: \"kubernetes.io/projected/765cc7b9-0dec-437a-9526-42d297733413-kube-api-access-ctpkh\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:52:37.638617 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.638587    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/765cc7b9-0dec-437a-9526-42d297733413-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:52:37.638617 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.638595    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hjpn4\" (UniqueName: \"kubernetes.io/projected/f09cd497-de22-42bf-b970-c29c995b35ae-kube-api-access-hjpn4\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:52:37.758374 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.758353    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"]
Jul 25 01:52:37.761839 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.761820    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-320e6c-55dcb57574-2nths"]
Jul 25 01:52:37.771469 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.771443    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"]
Jul 25 01:52:37.774807 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:37.774788    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-320e6c-546796b84-86ps9"]
Jul 25 01:52:38.075875 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:38.075801    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="765cc7b9-0dec-437a-9526-42d297733413" path="/var/lib/kubelet/pods/765cc7b9-0dec-437a-9526-42d297733413/volumes"
Jul 25 01:52:38.076199 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:38.076185    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f09cd497-de22-42bf-b970-c29c995b35ae" path="/var/lib/kubelet/pods/f09cd497-de22-42bf-b970-c29c995b35ae/volumes"
Jul 25 01:52:41.173433 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:41.173392    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:41.549909 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:41.549827    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:47.397448 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:47.397416    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:47.498345 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:47.498309    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:47.674262 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:47.674173    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:47.898197 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:47.898159    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:48.257500 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:48.257468    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:48.572530 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:48.572450    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:48.708947 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:48.708918    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:51.139377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.139342    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:51.220562 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.220523    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:51.308679 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.308653    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:51.481482 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.481406    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:51.790254 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.790168    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:51.889286 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.889257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:51.890427 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.890383    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"]
Jul 25 01:52:51.890965 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.890945    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a7696e25-54f9-4e75-b25e-f99bb134f5ca" containerName="payload-processing"
Jul 25 01:52:51.890965 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.890965    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="a7696e25-54f9-4e75-b25e-f99bb134f5ca" containerName="payload-processing"
Jul 25 01:52:51.891091 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.890990    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f09cd497-de22-42bf-b970-c29c995b35ae" containerName="payload-processing"
Jul 25 01:52:51.891091 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.890997    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="f09cd497-de22-42bf-b970-c29c995b35ae" containerName="payload-processing"
Jul 25 01:52:51.891091 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.891006    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cf8aad7c-9d17-4366-b470-ba04368d6721" containerName="payload-pre-processing"
Jul 25 01:52:51.891091 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.891012    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="cf8aad7c-9d17-4366-b470-ba04368d6721" containerName="payload-pre-processing"
Jul 25 01:52:51.891091 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.891019    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="765cc7b9-0dec-437a-9526-42d297733413" containerName="payload-pre-processing"
Jul 25 01:52:51.891091 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.891025    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="765cc7b9-0dec-437a-9526-42d297733413" containerName="payload-pre-processing"
Jul 25 01:52:51.891091 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.891093    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="765cc7b9-0dec-437a-9526-42d297733413" containerName="payload-pre-processing"
Jul 25 01:52:51.891302 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.891102    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="cf8aad7c-9d17-4366-b470-ba04368d6721" containerName="payload-pre-processing"
Jul 25 01:52:51.891302 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.891107    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="f09cd497-de22-42bf-b970-c29c995b35ae" containerName="payload-processing"
Jul 25 01:52:51.891302 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.891113    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="a7696e25-54f9-4e75-b25e-f99bb134f5ca" containerName="payload-processing"
Jul 25 01:52:51.894389 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.894374    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:51.897195 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.897179    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-88f354-dockercfg-tkwh7\""
Jul 25 01:52:51.897302 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.897232    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-88f354\""
Jul 25 01:52:51.907336 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.907316    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"]
Jul 25 01:52:51.934000 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.933960    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"]
Jul 25 01:52:51.938027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.938001    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:51.944550 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.944530    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"]
Jul 25 01:52:51.963953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.963932    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg8d7\" (UniqueName: \"kubernetes.io/projected/9dcd741f-309d-4007-bfd6-9b8d706fe822-kube-api-access-vg8d7\") pod \"payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk\" (UID: \"9dcd741f-309d-4007-bfd6-9b8d706fe822\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:51.964063 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.964002    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd741f-309d-4007-bfd6-9b8d706fe822-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk\" (UID: \"9dcd741f-309d-4007-bfd6-9b8d706fe822\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:51.964063 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.964048    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dchd\" (UniqueName: \"kubernetes.io/projected/afe7259f-013e-4cf0-94b3-497b5287b01e-kube-api-access-6dchd\") pod \"payload-processing-e2e-api-a-88f354-85fc786bbd-zz985\" (UID: \"afe7259f-013e-4cf0-94b3-497b5287b01e\") " pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:51.964160 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:51.964122    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe7259f-013e-4cf0-94b3-497b5287b01e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-88f354-85fc786bbd-zz985\" (UID: \"afe7259f-013e-4cf0-94b3-497b5287b01e\") " pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:52.064660 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.064575    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg8d7\" (UniqueName: \"kubernetes.io/projected/9dcd741f-309d-4007-bfd6-9b8d706fe822-kube-api-access-vg8d7\") pod \"payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk\" (UID: \"9dcd741f-309d-4007-bfd6-9b8d706fe822\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:52.064660 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.064631    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd741f-309d-4007-bfd6-9b8d706fe822-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk\" (UID: \"9dcd741f-309d-4007-bfd6-9b8d706fe822\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:52.064660 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.064658    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dchd\" (UniqueName: \"kubernetes.io/projected/afe7259f-013e-4cf0-94b3-497b5287b01e-kube-api-access-6dchd\") pod \"payload-processing-e2e-api-a-88f354-85fc786bbd-zz985\" (UID: \"afe7259f-013e-4cf0-94b3-497b5287b01e\") " pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:52.064941 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.064720    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe7259f-013e-4cf0-94b3-497b5287b01e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-88f354-85fc786bbd-zz985\" (UID: \"afe7259f-013e-4cf0-94b3-497b5287b01e\") " pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:52.065386 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.065365    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd741f-309d-4007-bfd6-9b8d706fe822-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk\" (UID: \"9dcd741f-309d-4007-bfd6-9b8d706fe822\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:52.065435 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.065383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe7259f-013e-4cf0-94b3-497b5287b01e-plugins-config-volume\") pod \"payload-processing-e2e-api-a-88f354-85fc786bbd-zz985\" (UID: \"afe7259f-013e-4cf0-94b3-497b5287b01e\") " pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:52.073486 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.073464    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg8d7\" (UniqueName: \"kubernetes.io/projected/9dcd741f-309d-4007-bfd6-9b8d706fe822-kube-api-access-vg8d7\") pod \"payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk\" (UID: \"9dcd741f-309d-4007-bfd6-9b8d706fe822\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:52.073767 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.073745    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dchd\" (UniqueName: \"kubernetes.io/projected/afe7259f-013e-4cf0-94b3-497b5287b01e-kube-api-access-6dchd\") pod \"payload-processing-e2e-api-a-88f354-85fc786bbd-zz985\" (UID: \"afe7259f-013e-4cf0-94b3-497b5287b01e\") " pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:52.204704 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.204672    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:52.249797 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.249769    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:52.347914 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.347320    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"]
Jul 25 01:52:52.349603 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:52:52.349567    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dcd741f_309d_4007_bfd6_9b8d706fe822.slice/crio-92dd33ac60fb02d1b5c7480a6741de2e01649a239246a8a65409c762ec7276ef WatchSource:0}: Error finding container 92dd33ac60fb02d1b5c7480a6741de2e01649a239246a8a65409c762ec7276ef: Status 404 returned error can't find the container with id 92dd33ac60fb02d1b5c7480a6741de2e01649a239246a8a65409c762ec7276ef
Jul 25 01:52:52.401936 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.401899    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"]
Jul 25 01:52:52.403503 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:52:52.403477    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podafe7259f_013e_4cf0_94b3_497b5287b01e.slice/crio-ccd0e2ed539ef1b33b540fc47ad80c64dc094d90e85267f760b1a758f473a3ca WatchSource:0}: Error finding container ccd0e2ed539ef1b33b540fc47ad80c64dc094d90e85267f760b1a758f473a3ca: Status 404 returned error can't find the container with id ccd0e2ed539ef1b33b540fc47ad80c64dc094d90e85267f760b1a758f473a3ca
Jul 25 01:52:52.520479 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.520441    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985" event={"ID":"afe7259f-013e-4cf0-94b3-497b5287b01e","Type":"ContainerStarted","Data":"8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38"}
Jul 25 01:52:52.520661 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.520504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985" event={"ID":"afe7259f-013e-4cf0-94b3-497b5287b01e","Type":"ContainerStarted","Data":"ccd0e2ed539ef1b33b540fc47ad80c64dc094d90e85267f760b1a758f473a3ca"}
Jul 25 01:52:52.520661 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.520534    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:52:52.522222 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.522197    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk" event={"ID":"9dcd741f-309d-4007-bfd6-9b8d706fe822","Type":"ContainerStarted","Data":"a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b"}
Jul 25 01:52:52.522347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.522284    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:52:52.522347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.522311    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk" event={"ID":"9dcd741f-309d-4007-bfd6-9b8d706fe822","Type":"ContainerStarted","Data":"92dd33ac60fb02d1b5c7480a6741de2e01649a239246a8a65409c762ec7276ef"}
Jul 25 01:52:52.536801 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:52.536684    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985" podStartSLOduration=1.5366584429999999 podStartE2EDuration="1.536658443s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:52:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:52:52.535201925 +0000 UTC m=+2451.006158265" watchObservedRunningTime="2026-07-25 01:52:52.536658443 +0000 UTC m=+2451.007614783"
Jul 25 01:52:55.642412 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:55.642380    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:56.105478 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.105388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:56.385643 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.385578    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk" podStartSLOduration=5.385560926 podStartE2EDuration="5.385560926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:52:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:52:52.552238219 +0000 UTC m=+2451.023194557" watchObservedRunningTime="2026-07-25 01:52:56.385560926 +0000 UTC m=+2454.856517266"
Jul 25 01:52:56.386245 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.386223    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"]
Jul 25 01:52:56.390265 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.390247    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.392733 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.392715    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-393e3a-openshift-default-dockercfg-9fg8x\""
Jul 25 01:52:56.393657 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393636    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mlrd\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-kube-api-access-4mlrd\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.393768 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393679    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-socket\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.393768 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393698    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-token\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.393768 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393732    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-data\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.393768 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393753    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-podinfo\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.394053 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393825    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istiod-ca-cert\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.394053 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393859    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-credential-socket\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.394053 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393900    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-certs\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.394053 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.393938    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-envoy\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.400395 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.400361    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"]
Jul 25 01:52:56.412760 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.412733    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:56.494780 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.494749    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-socket\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.494780 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.494774    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-token\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.494953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.494805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-data\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.494953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.494821    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-podinfo\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.494953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.494847    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istiod-ca-cert\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.494953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.494873    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-credential-socket\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.494953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.494907    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-certs\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.494953 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.494954    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-envoy\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.495316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.495036    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mlrd\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-kube-api-access-4mlrd\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.495316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.495214    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-socket\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.495316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.495244    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-data\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.495494 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.495475    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-certs\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.495564 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.495545    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-credential-socket\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.495780 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.495758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istiod-ca-cert\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.497139 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.497113    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-envoy\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.497501 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.497481    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-podinfo\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.502589 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.502568    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-token\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.503016 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.502959    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mlrd\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-kube-api-access-4mlrd\") pod \"e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") " pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.514023 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.513995    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:56.702267 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.702188    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:56.840106 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.840083    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"]
Jul 25 01:52:56.843191 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:52:56.843158    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667f1638_e02f_4a93_8a0b_d13fd9a19b58.slice/crio-172bab721edced7abbd790528075071392c88a4cdfd114642955995922b2c7df WatchSource:0}: Error finding container 172bab721edced7abbd790528075071392c88a4cdfd114642955995922b2c7df: Status 404 returned error can't find the container with id 172bab721edced7abbd790528075071392c88a4cdfd114642955995922b2c7df
Jul 25 01:52:56.846077 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.846042    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:52:56.846175 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.846121    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:52:56.846175 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:56.846165    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:52:57.548469 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:57.548430    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" event={"ID":"667f1638-e02f-4a93-8a0b-d13fd9a19b58","Type":"ContainerStarted","Data":"0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c"}
Jul 25 01:52:57.548677 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:57.548493    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" event={"ID":"667f1638-e02f-4a93-8a0b-d13fd9a19b58","Type":"ContainerStarted","Data":"172bab721edced7abbd790528075071392c88a4cdfd114642955995922b2c7df"}
Jul 25 01:52:57.568312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:57.568272    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" podStartSLOduration=1.568258743 podStartE2EDuration="1.568258743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:52:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:52:57.566847485 +0000 UTC m=+2456.037803824" watchObservedRunningTime="2026-07-25 01:52:57.568258743 +0000 UTC m=+2456.039215081"
Jul 25 01:52:57.702298 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:57.702273    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:57.707171 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:57.707145    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:58.561217 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:58.561181    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:58.562078 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:58.562054    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:52:59.452855 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:59.452822    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:59.532947 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:59.532912    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:59.611399 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:59.611370    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:52:59.740442 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:52:59.740362    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:00.131488 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.131441    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:00.221234 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.221206    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:00.259679 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.259650    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"]
Jul 25 01:53:00.263229 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.263214    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:00.267139 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.267121    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-393e3a-dockercfg-bmxmj\""
Jul 25 01:53:00.267387 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.267368    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-393e3a\""
Jul 25 01:53:00.277274 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.277253    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"]
Jul 25 01:53:00.328022 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.327954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2kjrg\" (UniqueName: \"kubernetes.io/projected/ab1b6488-0031-4a59-845f-6e4bdf0defc2-kube-api-access-2kjrg\") pod \"payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv\" (UID: \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\") " pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:00.328127 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.328098    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1b6488-0031-4a59-845f-6e4bdf0defc2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv\" (UID: \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\") " pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:00.429198 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.429113    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2kjrg\" (UniqueName: \"kubernetes.io/projected/ab1b6488-0031-4a59-845f-6e4bdf0defc2-kube-api-access-2kjrg\") pod \"payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv\" (UID: \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\") " pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:00.429198 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.429176    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1b6488-0031-4a59-845f-6e4bdf0defc2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv\" (UID: \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\") " pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:00.429850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.429832    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1b6488-0031-4a59-845f-6e4bdf0defc2-plugins-config-volume\") pod \"payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv\" (UID: \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\") " pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:00.439691 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.439649    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2kjrg\" (UniqueName: \"kubernetes.io/projected/ab1b6488-0031-4a59-845f-6e4bdf0defc2-kube-api-access-2kjrg\") pod \"payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv\" (UID: \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\") " pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:00.574653 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.574620    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:00.930897 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:00.930871    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"]
Jul 25 01:53:00.932932 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:53:00.932906    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab1b6488_0031_4a59_845f_6e4bdf0defc2.slice/crio-377871f360cbf6500b3e88c086df415dff5e9ed2fb5016ab3d48fd79164f783e WatchSource:0}: Error finding container 377871f360cbf6500b3e88c086df415dff5e9ed2fb5016ab3d48fd79164f783e: Status 404 returned error can't find the container with id 377871f360cbf6500b3e88c086df415dff5e9ed2fb5016ab3d48fd79164f783e
Jul 25 01:53:01.577390 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:01.577357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv" event={"ID":"ab1b6488-0031-4a59-845f-6e4bdf0defc2","Type":"ContainerStarted","Data":"6fb7b84d3f1fde9f39ef254083ff1119afc6c434ec615dda979fe14e75b9765f"}
Jul 25 01:53:01.577843 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:01.577410    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv" event={"ID":"ab1b6488-0031-4a59-845f-6e4bdf0defc2","Type":"ContainerStarted","Data":"377871f360cbf6500b3e88c086df415dff5e9ed2fb5016ab3d48fd79164f783e"}
Jul 25 01:53:01.577843 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:01.577435    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:01.593683 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:01.593637    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv" podStartSLOduration=1.5936191929999999 podStartE2EDuration="1.593619193s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:53:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:53:01.593044635 +0000 UTC m=+2460.064000973" watchObservedRunningTime="2026-07-25 01:53:01.593619193 +0000 UTC m=+2460.064575509"
Jul 25 01:53:03.529459 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:03.529429    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:53:03.529838 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:03.529589    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:53:03.967512 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:03.967474    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:12.585167 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:12.585141    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:14.603138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:14.603095    2562 patch_prober.go:28] interesting pod/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 25 01:53:14.603535 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:14.603161    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 25 01:53:15.653126 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:15.653097    2562 generic.go:356] "Generic (PLEG): container finished" podID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerID="0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c" exitCode=0
Jul 25 01:53:15.653503 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:15.653149    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" event={"ID":"667f1638-e02f-4a93-8a0b-d13fd9a19b58","Type":"ContainerDied","Data":"0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c"}
Jul 25 01:53:15.653549 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:15.653510    2562 scope.go:117] "RemoveContainer" containerID="0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c"
Jul 25 01:53:15.654525 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:15.654500    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:53:15.654579 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:15.654560    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:53:15.654615 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:15.654590    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:53:16.663547 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:16.663504    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" event={"ID":"667f1638-e02f-4a93-8a0b-d13fd9a19b58","Type":"ContainerStarted","Data":"2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc"}
Jul 25 01:53:16.702364 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:16.702332    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:53:16.702563 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:16.702542    2562 patch_prober.go:28] interesting pod/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 25 01:53:16.702615 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:16.702586    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 25 01:53:17.702726 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:17.702687    2562 patch_prober.go:28] interesting pod/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 25 01:53:17.703211 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:17.702761    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 25 01:53:18.702758 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:18.702717    2562 patch_prober.go:28] interesting pod/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 25 01:53:18.703337 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:18.702789    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 25 01:53:19.703293 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:19.703257    2562 patch_prober.go:28] interesting pod/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused" start-of-body=
Jul 25 01:53:19.703670 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:19.703320    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.64:15021/healthz/ready\": dial tcp 10.134.0.64:15021: connect: connection refused"
Jul 25 01:53:20.706713 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:20.706685    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:53:20.707091 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:20.706942    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:53:20.707597 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:20.707579    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:53:27.846893 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:27.846857    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:27.875730 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:27.875702    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"]
Jul 25 01:53:27.875925 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:27.875902    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985" podUID="afe7259f-013e-4cf0-94b3-497b5287b01e" containerName="payload-processing" containerID="cri-o://8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38" gracePeriod=30
Jul 25 01:53:27.881027 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:27.881000    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"]
Jul 25 01:53:27.881245 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:27.881220    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk" podUID="9dcd741f-309d-4007-bfd6-9b8d706fe822" containerName="payload-pre-processing" containerID="cri-o://a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b" gracePeriod=30
Jul 25 01:53:27.917871 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:27.917829    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:27.972669 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:27.972642    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:28.141012 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.140958    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:53:28.199675 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.199655    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:53:28.281573 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.281548    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vg8d7\" (UniqueName: \"kubernetes.io/projected/9dcd741f-309d-4007-bfd6-9b8d706fe822-kube-api-access-vg8d7\") pod \"9dcd741f-309d-4007-bfd6-9b8d706fe822\" (UID: \"9dcd741f-309d-4007-bfd6-9b8d706fe822\") "
Jul 25 01:53:28.281747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.281644    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dchd\" (UniqueName: \"kubernetes.io/projected/afe7259f-013e-4cf0-94b3-497b5287b01e-kube-api-access-6dchd\") pod \"afe7259f-013e-4cf0-94b3-497b5287b01e\" (UID: \"afe7259f-013e-4cf0-94b3-497b5287b01e\") "
Jul 25 01:53:28.281747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.281691    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd741f-309d-4007-bfd6-9b8d706fe822-plugins-config-volume\") pod \"9dcd741f-309d-4007-bfd6-9b8d706fe822\" (UID: \"9dcd741f-309d-4007-bfd6-9b8d706fe822\") "
Jul 25 01:53:28.281747 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.281723    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe7259f-013e-4cf0-94b3-497b5287b01e-plugins-config-volume\") pod \"afe7259f-013e-4cf0-94b3-497b5287b01e\" (UID: \"afe7259f-013e-4cf0-94b3-497b5287b01e\") "
Jul 25 01:53:28.282116 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.282090    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9dcd741f-309d-4007-bfd6-9b8d706fe822-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "9dcd741f-309d-4007-bfd6-9b8d706fe822" (UID: "9dcd741f-309d-4007-bfd6-9b8d706fe822"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:53:28.282232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.282202    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/afe7259f-013e-4cf0-94b3-497b5287b01e-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "afe7259f-013e-4cf0-94b3-497b5287b01e" (UID: "afe7259f-013e-4cf0-94b3-497b5287b01e"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:53:28.283613 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.283593    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dcd741f-309d-4007-bfd6-9b8d706fe822-kube-api-access-vg8d7" (OuterVolumeSpecName: "kube-api-access-vg8d7") pod "9dcd741f-309d-4007-bfd6-9b8d706fe822" (UID: "9dcd741f-309d-4007-bfd6-9b8d706fe822"). InnerVolumeSpecName "kube-api-access-vg8d7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:53:28.283745 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.283727    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/afe7259f-013e-4cf0-94b3-497b5287b01e-kube-api-access-6dchd" (OuterVolumeSpecName: "kube-api-access-6dchd") pod "afe7259f-013e-4cf0-94b3-497b5287b01e" (UID: "afe7259f-013e-4cf0-94b3-497b5287b01e"). InnerVolumeSpecName "kube-api-access-6dchd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:53:28.382585 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.382563    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dchd\" (UniqueName: \"kubernetes.io/projected/afe7259f-013e-4cf0-94b3-497b5287b01e-kube-api-access-6dchd\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:53:28.382585 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.382586    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9dcd741f-309d-4007-bfd6-9b8d706fe822-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:53:28.382743 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.382596    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/afe7259f-013e-4cf0-94b3-497b5287b01e-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:53:28.382743 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.382606    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg8d7\" (UniqueName: \"kubernetes.io/projected/9dcd741f-309d-4007-bfd6-9b8d706fe822-kube-api-access-vg8d7\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:53:28.730353 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.730271    2562 generic.go:356] "Generic (PLEG): container finished" podID="afe7259f-013e-4cf0-94b3-497b5287b01e" containerID="8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38" exitCode=0
Jul 25 01:53:28.730353 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.730332    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"
Jul 25 01:53:28.730583 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.730357    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985" event={"ID":"afe7259f-013e-4cf0-94b3-497b5287b01e","Type":"ContainerDied","Data":"8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38"}
Jul 25 01:53:28.730583 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.730440    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985" event={"ID":"afe7259f-013e-4cf0-94b3-497b5287b01e","Type":"ContainerDied","Data":"ccd0e2ed539ef1b33b540fc47ad80c64dc094d90e85267f760b1a758f473a3ca"}
Jul 25 01:53:28.730583 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.730463    2562 scope.go:117] "RemoveContainer" containerID="8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38"
Jul 25 01:53:28.731751 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.731721    2562 generic.go:356] "Generic (PLEG): container finished" podID="9dcd741f-309d-4007-bfd6-9b8d706fe822" containerID="a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b" exitCode=0
Jul 25 01:53:28.731868 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.731783    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk" event={"ID":"9dcd741f-309d-4007-bfd6-9b8d706fe822","Type":"ContainerDied","Data":"a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b"}
Jul 25 01:53:28.731868 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.731792    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"
Jul 25 01:53:28.731868 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.731825    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk" event={"ID":"9dcd741f-309d-4007-bfd6-9b8d706fe822","Type":"ContainerDied","Data":"92dd33ac60fb02d1b5c7480a6741de2e01649a239246a8a65409c762ec7276ef"}
Jul 25 01:53:28.740632 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.740614    2562 scope.go:117] "RemoveContainer" containerID="8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38"
Jul 25 01:53:28.740924 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:53:28.740888    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38\": container with ID starting with 8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38 not found: ID does not exist" containerID="8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38"
Jul 25 01:53:28.741053 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.740925    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38"} err="failed to get container status \"8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38\": rpc error: code = NotFound desc = could not find container \"8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38\": container with ID starting with 8b50d4007d0eaeeec0e68536a73af074e4339e4e9262d532910dfa30f4cc2d38 not found: ID does not exist"
Jul 25 01:53:28.741053 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.740947    2562 scope.go:117] "RemoveContainer" containerID="a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b"
Jul 25 01:53:28.749783 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.749767    2562 scope.go:117] "RemoveContainer" containerID="a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b"
Jul 25 01:53:28.750047 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:53:28.750026    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b\": container with ID starting with a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b not found: ID does not exist" containerID="a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b"
Jul 25 01:53:28.750124 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.750055    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b"} err="failed to get container status \"a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b\": rpc error: code = NotFound desc = could not find container \"a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b\": container with ID starting with a580c2b30398338d8c0ba88ffcc0c1a2062f8e3fb7135314c4a64d6a2eb2772b not found: ID does not exist"
Jul 25 01:53:28.756370 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.756349    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"]
Jul 25 01:53:28.760092 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.760073    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-88f354-85fc786bbd-zz985"]
Jul 25 01:53:28.769637 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.769616    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"]
Jul 25 01:53:28.773301 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:28.773276    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-88f354-7544855dfb-69vmk"]
Jul 25 01:53:30.076917 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:30.076889    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dcd741f-309d-4007-bfd6-9b8d706fe822" path="/var/lib/kubelet/pods/9dcd741f-309d-4007-bfd6-9b8d706fe822/volumes"
Jul 25 01:53:30.077310 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:30.077293    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="afe7259f-013e-4cf0-94b3-497b5287b01e" path="/var/lib/kubelet/pods/afe7259f-013e-4cf0-94b3-497b5287b01e/volumes"
Jul 25 01:53:32.270049 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:32.269929    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:32.667930 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:32.667883    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:38.527756 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:38.527677    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:38.629493 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:38.629460    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:38.800066 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:38.799961    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:39.033635 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:39.033595    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:54.698963 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:54.698845    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:54.725394 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:54.725359    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"]
Jul 25 01:53:54.725685 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:54.725628    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv" podUID="ab1b6488-0031-4a59-845f-6e4bdf0defc2" containerName="payload-processing" containerID="cri-o://6fb7b84d3f1fde9f39ef254083ff1119afc6c434ec615dda979fe14e75b9765f" gracePeriod=30
Jul 25 01:53:54.769502 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:54.769476    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:54.863531 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:54.863504    2562 generic.go:356] "Generic (PLEG): container finished" podID="ab1b6488-0031-4a59-845f-6e4bdf0defc2" containerID="6fb7b84d3f1fde9f39ef254083ff1119afc6c434ec615dda979fe14e75b9765f" exitCode=0
Jul 25 01:53:54.863651 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:54.863563    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv" event={"ID":"ab1b6488-0031-4a59-845f-6e4bdf0defc2","Type":"ContainerDied","Data":"6fb7b84d3f1fde9f39ef254083ff1119afc6c434ec615dda979fe14e75b9765f"}
Jul 25 01:53:54.869824 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:54.869804    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:54.994224 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:54.994196    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:55.094072 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.094042    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1b6488-0031-4a59-845f-6e4bdf0defc2-plugins-config-volume\") pod \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\" (UID: \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\") "
Jul 25 01:53:55.094232 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.094099    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2kjrg\" (UniqueName: \"kubernetes.io/projected/ab1b6488-0031-4a59-845f-6e4bdf0defc2-kube-api-access-2kjrg\") pod \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\" (UID: \"ab1b6488-0031-4a59-845f-6e4bdf0defc2\") "
Jul 25 01:53:55.094411 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.094391    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab1b6488-0031-4a59-845f-6e4bdf0defc2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab1b6488-0031-4a59-845f-6e4bdf0defc2" (UID: "ab1b6488-0031-4a59-845f-6e4bdf0defc2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:53:55.095934 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.095912    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab1b6488-0031-4a59-845f-6e4bdf0defc2-kube-api-access-2kjrg" (OuterVolumeSpecName: "kube-api-access-2kjrg") pod "ab1b6488-0031-4a59-845f-6e4bdf0defc2" (UID: "ab1b6488-0031-4a59-845f-6e4bdf0defc2"). InnerVolumeSpecName "kube-api-access-2kjrg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:53:55.194784 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.194759    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab1b6488-0031-4a59-845f-6e4bdf0defc2-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:53:55.194784 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.194780    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2kjrg\" (UniqueName: \"kubernetes.io/projected/ab1b6488-0031-4a59-845f-6e4bdf0defc2-kube-api-access-2kjrg\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:53:55.869606 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.869570    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"
Jul 25 01:53:55.870026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.869605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv" event={"ID":"ab1b6488-0031-4a59-845f-6e4bdf0defc2","Type":"ContainerDied","Data":"377871f360cbf6500b3e88c086df415dff5e9ed2fb5016ab3d48fd79164f783e"}
Jul 25 01:53:55.870026 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.869662    2562 scope.go:117] "RemoveContainer" containerID="6fb7b84d3f1fde9f39ef254083ff1119afc6c434ec615dda979fe14e75b9765f"
Jul 25 01:53:55.890295 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.890265    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"]
Jul 25 01:53:55.893912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:55.893892    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-393e3a-86cc879ffd-79knv"]
Jul 25 01:53:56.076002 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:56.075960    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab1b6488-0031-4a59-845f-6e4bdf0defc2" path="/var/lib/kubelet/pods/ab1b6488-0031-4a59-845f-6e4bdf0defc2/volumes"
Jul 25 01:53:59.244297 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:59.244270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:53:59.649729 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:53:59.649700    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:05.163912 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:05.163880    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:05.264402 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:05.264369    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:05.417941 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:05.417857    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"]
Jul 25 01:54:05.418158 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:05.418133    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy" containerID="cri-o://2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc" gracePeriod=30
Jul 25 01:54:05.432322 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:05.432285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:05.675850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:05.675770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:05.708687 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:05.708660    2562 patch_prober.go:28] interesting pod/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Jul 25 01:54:05.708816 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:05.708716    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Jul 25 01:54:06.023850 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:06.023768    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:06.327415 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:06.327344    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:06.484097 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:06.484063    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:09.576280 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:09.576239    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:09.647299 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:09.647263    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:09.713079 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:09.713053    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:09.936671 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:09.936631    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:10.030712 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.030675    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:10.292502 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.292404    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:10.387186 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.387157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:10.689090 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.689068    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:54:10.721751 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.721727    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-certs\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.721877 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.721783    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-envoy\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.721877 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.721816    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-data\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.721877 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.721840    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-token\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.721877 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.721864    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-credential-socket\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.722138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.721938    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mlrd\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-kube-api-access-4mlrd\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.722138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.721989    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-socket\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.722138 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.722034    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:54:10.722300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.722144    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istiod-ca-cert\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.722300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.722189    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-podinfo\") pod \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\" (UID: \"667f1638-e02f-4a93-8a0b-d13fd9a19b58\") "
Jul 25 01:54:10.722300 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.722285    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:54:10.722741 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.722522    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-certs\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.722741 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.722547    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-workload-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.722741 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.722549    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 01:54:10.722741 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.722695    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:54:10.723347 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.723317    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-data" (OuterVolumeSpecName: "istio-data") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:54:10.724309 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.724282    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 01:54:10.724418 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.724398    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 01:54:10.724476 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.724416    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-kube-api-access-4mlrd" (OuterVolumeSpecName: "kube-api-access-4mlrd") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "kube-api-access-4mlrd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:54:10.724476 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.724447    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-token" (OuterVolumeSpecName: "istio-token") pod "667f1638-e02f-4a93-8a0b-d13fd9a19b58" (UID: "667f1638-e02f-4a93-8a0b-d13fd9a19b58"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 01:54:10.823372 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.823344    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-envoy\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.823372 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.823369    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-data\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.823372 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.823378    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-token\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.823587 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.823387    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/667f1638-e02f-4a93-8a0b-d13fd9a19b58-credential-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.823587 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.823396    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mlrd\" (UniqueName: \"kubernetes.io/projected/667f1638-e02f-4a93-8a0b-d13fd9a19b58-kube-api-access-4mlrd\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.823587 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.823405    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istiod-ca-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.823587 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.823413    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/667f1638-e02f-4a93-8a0b-d13fd9a19b58-istio-podinfo\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 01:54:10.942547 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.942508    2562 generic.go:356] "Generic (PLEG): container finished" podID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerID="2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc" exitCode=0
Jul 25 01:54:10.942738 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.942562    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"
Jul 25 01:54:10.942738 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.942605    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" event={"ID":"667f1638-e02f-4a93-8a0b-d13fd9a19b58","Type":"ContainerDied","Data":"2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc"}
Jul 25 01:54:10.942738 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.942713    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w" event={"ID":"667f1638-e02f-4a93-8a0b-d13fd9a19b58","Type":"ContainerDied","Data":"172bab721edced7abbd790528075071392c88a4cdfd114642955995922b2c7df"}
Jul 25 01:54:10.942905 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.942747    2562 scope.go:117] "RemoveContainer" containerID="2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc"
Jul 25 01:54:10.953349 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.953331    2562 scope.go:117] "RemoveContainer" containerID="0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c"
Jul 25 01:54:10.960811 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.960793    2562 scope.go:117] "RemoveContainer" containerID="2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc"
Jul 25 01:54:10.961071 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:54:10.961053    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc\": container with ID starting with 2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc not found: ID does not exist" containerID="2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc"
Jul 25 01:54:10.961159 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.961084    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc"} err="failed to get container status \"2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc\": rpc error: code = NotFound desc = could not find container \"2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc\": container with ID starting with 2c03582ee287763c7b584c5f19999a00428be8b5b3d779b53a7f35b1804649cc not found: ID does not exist"
Jul 25 01:54:10.961159 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.961107    2562 scope.go:117] "RemoveContainer" containerID="0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c"
Jul 25 01:54:10.961361 ip-10-0-129-103 kubenswrapper[2562]: E0725 01:54:10.961343    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c\": container with ID starting with 0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c not found: ID does not exist" containerID="0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c"
Jul 25 01:54:10.961398 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.961369    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c"} err="failed to get container status \"0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c\": rpc error: code = NotFound desc = could not find container \"0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c\": container with ID starting with 0eccc1018289534283a4d1ee1a5183078b4509a27518111b656559f9bad22b5c not found: ID does not exist"
Jul 25 01:54:10.965197 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.965176    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"]
Jul 25 01:54:10.970727 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:10.970709    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-393e3a-openshift-default-65775d6674-wrs4w"]
Jul 25 01:54:12.076271 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:12.076239    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" path="/var/lib/kubelet/pods/667f1638-e02f-4a93-8a0b-d13fd9a19b58/volumes"
Jul 25 01:54:14.111311 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.111283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:14.447080 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.447026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:14.747623 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.747547    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"]
Jul 25 01:54:14.747940 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.747928    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="afe7259f-013e-4cf0-94b3-497b5287b01e" containerName="payload-processing"
Jul 25 01:54:14.748014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.747942    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="afe7259f-013e-4cf0-94b3-497b5287b01e" containerName="payload-processing"
Jul 25 01:54:14.748014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.747955    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy"
Jul 25 01:54:14.748014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.747960    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy"
Jul 25 01:54:14.748014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.747966    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab1b6488-0031-4a59-845f-6e4bdf0defc2" containerName="payload-processing"
Jul 25 01:54:14.748014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.747971    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab1b6488-0031-4a59-845f-6e4bdf0defc2" containerName="payload-processing"
Jul 25 01:54:14.748014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.748002    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dcd741f-309d-4007-bfd6-9b8d706fe822" containerName="payload-pre-processing"
Jul 25 01:54:14.748014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.748007    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dcd741f-309d-4007-bfd6-9b8d706fe822" containerName="payload-pre-processing"
Jul 25 01:54:14.748268 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.748080    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy"
Jul 25 01:54:14.748268 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.748089    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dcd741f-309d-4007-bfd6-9b8d706fe822" containerName="payload-pre-processing"
Jul 25 01:54:14.748268 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.748095    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab1b6488-0031-4a59-845f-6e4bdf0defc2" containerName="payload-processing"
Jul 25 01:54:14.748268 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.748101    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="afe7259f-013e-4cf0-94b3-497b5287b01e" containerName="payload-processing"
Jul 25 01:54:14.748268 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.748107    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy"
Jul 25 01:54:14.752589 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.752570    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.755364 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.755341    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-ab0ae6-openshift-default-dockercfg-n68k2\""
Jul 25 01:54:14.763619 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.763595    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"]
Jul 25 01:54:14.793127 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.793091    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:14.857665 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.857626    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-socket\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.857665 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.857661    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-envoy\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.857889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.857727    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-credential-socket\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.857889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.857792    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-token\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.857889 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.857836    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dksb\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-kube-api-access-8dksb\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.858087 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.857905    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istiod-ca-cert\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.858087 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.857958    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-podinfo\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.858087 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.858026    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-certs\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.858087 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.858050    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-data\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959335 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959302    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-credential-socket\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959494 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959345    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-token\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959494 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959434    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dksb\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-kube-api-access-8dksb\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959494 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959462    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istiod-ca-cert\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959494 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959493    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-podinfo\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959766 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959662    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-certs\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959766 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959692    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-data\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959766 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959716    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-credential-socket\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959943 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959775    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-socket\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.959943 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.959805    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-envoy\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.960105 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.960070    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-data\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.960177 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.960154    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-certs\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.960357 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.960334    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-socket\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.960435 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.960410    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istiod-ca-cert\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.961931 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.961910    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-envoy\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.962184 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.962167    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-podinfo\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.966722 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.966702    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-token\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:14.966893 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:14.966878    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dksb\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-kube-api-access-8dksb\") pod \"e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") " pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:15.064774 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.064710    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:15.202171 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.202149    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"]
Jul 25 01:54:15.204159 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:54:15.204125    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8af909ce_3356_47c4_8e35_2a9ad7e3a194.slice/crio-3a0faf1e615a14124266e55b01faa1a939026c5f211eee16848e189ce22d2e4f WatchSource:0}: Error finding container 3a0faf1e615a14124266e55b01faa1a939026c5f211eee16848e189ce22d2e4f: Status 404 returned error can't find the container with id 3a0faf1e615a14124266e55b01faa1a939026c5f211eee16848e189ce22d2e4f
Jul 25 01:54:15.206212 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.206184    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:54:15.206289 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.206249    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:54:15.206289 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.206280    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:54:15.616471 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.616437    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:15.970621 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.970591    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" event={"ID":"8af909ce-3356-47c4-8e35-2a9ad7e3a194","Type":"ContainerStarted","Data":"a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032"}
Jul 25 01:54:15.970788 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.970640    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" event={"ID":"8af909ce-3356-47c4-8e35-2a9ad7e3a194","Type":"ContainerStarted","Data":"3a0faf1e615a14124266e55b01faa1a939026c5f211eee16848e189ce22d2e4f"}
Jul 25 01:54:15.993173 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:15.993125    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" podStartSLOduration=1.993110314 podStartE2EDuration="1.993110314s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:54:15.992450852 +0000 UTC m=+2534.463407200" watchObservedRunningTime="2026-07-25 01:54:15.993110314 +0000 UTC m=+2534.464066653"
Jul 25 01:54:16.065229 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:16.065201    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:16.079153 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:16.079130    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:16.976152 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:16.976123    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:16.976897 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:16.976880    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:17.981382 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:17.981351    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:18.080696 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.080665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:18.220291 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.220257    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:18.435693 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.435668    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:18.749773 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.749692    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:18.839634 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.839601    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:18.863410 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.863383    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"]
Jul 25 01:54:18.863793 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.863781    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy"
Jul 25 01:54:18.863841 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.863795    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="667f1638-e02f-4a93-8a0b-d13fd9a19b58" containerName="istio-proxy"
Jul 25 01:54:18.867286 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.867266    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:18.870148 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.870129    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-ab0ae6\""
Jul 25 01:54:18.870377 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.870363    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-ab0ae6-dockercfg-qw5zc\""
Jul 25 01:54:18.879242 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.879219    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"]
Jul 25 01:54:18.993909 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.993879    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9gmg\" (UniqueName: \"kubernetes.io/projected/82f78176-fe45-4655-b5b0-2e04afc9ea1b-kube-api-access-j9gmg\") pod \"payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d\" (UID: \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:18.994327 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:18.994017    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82f78176-fe45-4655-b5b0-2e04afc9ea1b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d\" (UID: \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:19.094711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.094633    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82f78176-fe45-4655-b5b0-2e04afc9ea1b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d\" (UID: \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:19.094711 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.094686    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9gmg\" (UniqueName: \"kubernetes.io/projected/82f78176-fe45-4655-b5b0-2e04afc9ea1b-kube-api-access-j9gmg\") pod \"payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d\" (UID: \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:19.095404 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.095383    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82f78176-fe45-4655-b5b0-2e04afc9ea1b-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d\" (UID: \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:19.102712 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.102679    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9gmg\" (UniqueName: \"kubernetes.io/projected/82f78176-fe45-4655-b5b0-2e04afc9ea1b-kube-api-access-j9gmg\") pod \"payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d\" (UID: \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\") " pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:19.180679 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.180652    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:19.318182 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.318159    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"]
Jul 25 01:54:19.319526 ip-10-0-129-103 kubenswrapper[2562]: W0725 01:54:19.319502    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f78176_fe45_4655_b5b0_2e04afc9ea1b.slice/crio-02f550d7f4b6fb8ebd0b647b3338d3d061aea34d66dfbfcad7bbd4d6cd3541a2 WatchSource:0}: Error finding container 02f550d7f4b6fb8ebd0b647b3338d3d061aea34d66dfbfcad7bbd4d6cd3541a2: Status 404 returned error can't find the container with id 02f550d7f4b6fb8ebd0b647b3338d3d061aea34d66dfbfcad7bbd4d6cd3541a2
Jul 25 01:54:19.994324 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.994281    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d" event={"ID":"82f78176-fe45-4655-b5b0-2e04afc9ea1b","Type":"ContainerStarted","Data":"510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320"}
Jul 25 01:54:19.994754 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.994334    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d" event={"ID":"82f78176-fe45-4655-b5b0-2e04afc9ea1b","Type":"ContainerStarted","Data":"02f550d7f4b6fb8ebd0b647b3338d3d061aea34d66dfbfcad7bbd4d6cd3541a2"}
Jul 25 01:54:19.994754 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:19.994419    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:20.011819 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:20.011776    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d" podStartSLOduration=2.011762555 podStartE2EDuration="2.011762555s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 01:54:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 01:54:20.011116962 +0000 UTC m=+2538.482073303" watchObservedRunningTime="2026-07-25 01:54:20.011762555 +0000 UTC m=+2538.482718946"
Jul 25 01:54:22.582155 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:22.582120    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:31.002110 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:31.002082    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 01:54:32.441420 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:32.441369    2562 patch_prober.go:28] interesting pod/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": read tcp 10.134.0.2:43090->10.134.0.66:15021: read: connection reset by peer" start-of-body=
Jul 25 01:54:32.441962 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:32.441499    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": read tcp 10.134.0.2:43090->10.134.0.66:15021: read: connection reset by peer"
Jul 25 01:54:33.064463 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:33.064429    2562 generic.go:356] "Generic (PLEG): container finished" podID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerID="a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032" exitCode=0
Jul 25 01:54:33.064630 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:33.064501    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" event={"ID":"8af909ce-3356-47c4-8e35-2a9ad7e3a194","Type":"ContainerDied","Data":"a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032"}
Jul 25 01:54:33.064897 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:33.064884    2562 scope.go:117] "RemoveContainer" containerID="a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032"
Jul 25 01:54:33.066014 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:33.065967    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:54:33.066102 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:33.066050    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:54:33.066102 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:33.066083    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 01:54:34.076421 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:34.076384    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" event={"ID":"8af909ce-3356-47c4-8e35-2a9ad7e3a194","Type":"ContainerStarted","Data":"6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87"}
Jul 25 01:54:34.132884 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:34.132855    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:35.065969 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:35.065940    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:35.066316 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:35.066295    2562 patch_prober.go:28] interesting pod/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 25 01:54:35.066479 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:35.066456    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 25 01:54:36.065752 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:36.065713    2562 patch_prober.go:28] interesting pod/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 25 01:54:36.066290 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:36.065781    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 25 01:54:36.977135 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:36.977098    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:37.065656 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:37.065616    2562 patch_prober.go:28] interesting pod/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused" start-of-body=
Jul 25 01:54:37.065832 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:37.065679    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.66:15021/healthz/ready\": dial tcp 10.134.0.66:15021: connect: connection refused"
Jul 25 01:54:37.078022 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:37.077991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:54:38.068719 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:38.068692    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:38.069170 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:38.069076    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:38.075721 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:38.075700    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 01:54:59.568797 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:54:59.568761    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:55:27.356721 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:55:27.356650    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:56:14.449808 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:56:14.449770    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:57:30.856861 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:57:30.856808    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:57:32.040312 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:57:32.040268    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:59:32.085957 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:59:32.085930    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:59:33.019536 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:59:33.019501    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 01:59:56.627631 ip-10-0-129-103 kubenswrapper[2562]: I0725 01:59:56.627552    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:00:14.403778 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:00:14.403745    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:00:14.452016 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:00:14.451963    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:00:28.915111 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:00:28.915075    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:01:11.995290 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:01:11.995256    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:01:58.022947 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:01:58.022908    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:01:59.016712 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:01:59.016680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:02:05.844319 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:02:05.844270    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:02:22.622131 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:02:22.622096    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:03:10.622316 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:03:10.622285    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:25.843823 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:25.843727    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:26.796419 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:26.796388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:46.717760 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:46.717724    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:47.048256 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:47.048163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:47.761886 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:47.761854    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:50.063196 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:50.063169    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:50.155286 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:50.155252    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:50.256513 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:50.256483    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:50.394869 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:50.394843    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:50.831069 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:50.830991    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:50.920420 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:50.920394    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:04:54.605736 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:04:54.605702    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:05.646710 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:05.646671    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:05.949196 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:05.949107    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"]
Jul 25 02:05:05.955131 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:05.955099    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:05.957628 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:05.957595    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-d66e66-openshift-default-dockercfg-dmcbx\""
Jul 25 02:05:05.966553 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:05.966526    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"]
Jul 25 02:05:05.981471 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:05.981420    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:06.036812 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.036769    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-data\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.036812 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.036815    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-token\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.037099 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.036833    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-credential-socket\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.037099 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.036895    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368020b2-dbf6-46c2-a5e7-f28701b59955-istiod-ca-cert\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.037099 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.036935    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-certs\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.037099 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.036954    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8tbn\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-kube-api-access-z8tbn\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.037099 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.036973    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-envoy\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.037099 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.037062    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-socket\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.037099 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.037086    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-podinfo\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.137832 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.137802    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-podinfo\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138054 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.137883    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-data\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138054 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.137904    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-token\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138054 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.137924    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-credential-socket\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138054 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.137963    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368020b2-dbf6-46c2-a5e7-f28701b59955-istiod-ca-cert\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138295 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138117    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-certs\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138295 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138162    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8tbn\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-kube-api-access-z8tbn\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138295 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138195    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-envoy\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138295 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138248    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-socket\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138526 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138382    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-data\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138584 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138522    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-credential-socket\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138789 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138758    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-socket\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138910 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138834    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368020b2-dbf6-46c2-a5e7-f28701b59955-istiod-ca-cert\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.138910 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.138837    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-certs\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.142132 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.142109    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-envoy\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.142304 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.142284    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-podinfo\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.145641 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.145608    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8tbn\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-kube-api-access-z8tbn\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.145832 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.145814    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-token\") pod \"e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") " pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.171554 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.171530    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:06.266959 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.266877    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:06.409533 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.409511    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"]
Jul 25 02:05:06.411711 ip-10-0-129-103 kubenswrapper[2562]: W0725 02:05:06.411687    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod368020b2_dbf6_46c2_a5e7_f28701b59955.slice/crio-18c014c245b08cdc950f9ae85263433b67643728108a52549c6ea3fd2beb542b WatchSource:0}: Error finding container 18c014c245b08cdc950f9ae85263433b67643728108a52549c6ea3fd2beb542b: Status 404 returned error can't find the container with id 18c014c245b08cdc950f9ae85263433b67643728108a52549c6ea3fd2beb542b
Jul 25 02:05:06.413908 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.413878    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:05:06.414007 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.413944    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:05:06.414007 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:06.413991    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:05:07.121025 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:07.120993    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" event={"ID":"368020b2-dbf6-46c2-a5e7-f28701b59955","Type":"ContainerStarted","Data":"6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1"}
Jul 25 02:05:07.121428 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:07.121045    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" event={"ID":"368020b2-dbf6-46c2-a5e7-f28701b59955","Type":"ContainerStarted","Data":"18c014c245b08cdc950f9ae85263433b67643728108a52549c6ea3fd2beb542b"}
Jul 25 02:05:07.140256 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:07.140201    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" podStartSLOduration=2.140185273 podStartE2EDuration="2.140185273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:05:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:05:07.138623549 +0000 UTC m=+3185.609579931" watchObservedRunningTime="2026-07-25 02:05:07.140185273 +0000 UTC m=+3185.611141610"
Jul 25 02:05:07.267748 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:07.267715    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:07.273103 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:07.273080    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:08.126142 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:08.126116    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:08.127033 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:08.127015    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:09.220386 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:09.220318    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:09.295746 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:09.295686    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:09.374188 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:09.374157    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:09.810492 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:09.810455    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:09.909995 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:09.909946    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:09.987802 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:09.987767    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:13.683097 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:13.683067    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:05:24.167180 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:24.167134    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Jul 25 02:05:24.167573 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:24.167221    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Jul 25 02:05:26.217431 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:26.217396    2562 generic.go:356] "Generic (PLEG): container finished" podID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerID="6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1" exitCode=0
Jul 25 02:05:26.217838 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:26.217469    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" event={"ID":"368020b2-dbf6-46c2-a5e7-f28701b59955","Type":"ContainerDied","Data":"6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1"}
Jul 25 02:05:26.217941 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:26.217925    2562 scope.go:117] "RemoveContainer" containerID="6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1"
Jul 25 02:05:26.219135 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:26.219100    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:05:26.219215 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:26.219190    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:05:26.219263 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:26.219236    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:05:27.224415 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:27.224385    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" event={"ID":"368020b2-dbf6-46c2-a5e7-f28701b59955","Type":"ContainerStarted","Data":"a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3"}
Jul 25 02:05:27.268142 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:27.268112    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:27.268351 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:27.268329    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 25 02:05:27.268402 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:27.268378    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 25 02:05:28.268298 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:28.268262    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 25 02:05:28.268681 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:28.268319    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 25 02:05:29.267741 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:29.267327    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 25 02:05:29.267741 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:29.267388    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 25 02:05:30.267885 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:30.267845    2562 patch_prober.go:28] interesting pod/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused" start-of-body=
Jul 25 02:05:30.268295 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:30.267914    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.68:15021/healthz/ready\": dial tcp 10.134.0.68:15021: connect: connection refused"
Jul 25 02:05:31.127803 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:31.127775    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-j8tq2_48ed41f2-58a1-4f84-9c9c-82525a777da2/payload-processing/0.log"
Jul 25 02:05:31.271444 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:31.271416    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:31.271825 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:31.271804    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:31.272567 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:31.272548    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:05:34.134444 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:05:34.134415    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:28.677198 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:28.677168    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-j8tq2_48ed41f2-58a1-4f84-9c9c-82525a777da2/payload-processing/0.log"
Jul 25 02:06:29.381219 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.381178    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"]
Jul 25 02:06:29.385026 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.385003    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.387138 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.387119    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-a585ca-openshift-default-dockercfg-q7qpg\""
Jul 25 02:06:29.396269 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.396247    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"]
Jul 25 02:06:29.499647 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.499613    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-certs\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.499939 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.499673    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d059df-4258-48bf-802b-319a753fe612-istiod-ca-cert\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.499939 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.499704    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b522\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-kube-api-access-4b522\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.499939 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.499773    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-credential-socket\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.499939 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.499830    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-istio-token\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.499939 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.499858    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d059df-4258-48bf-802b-319a753fe612-istio-podinfo\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.499939 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.499919    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-envoy\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.500311 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.499988    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-data\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.500311 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.500019    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-socket\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601036 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.600971    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-envoy\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601226 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601065    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-data\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601226 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601088    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-socket\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601226 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601137    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-certs\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601226 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601186    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d059df-4258-48bf-802b-319a753fe612-istiod-ca-cert\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601226 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601211    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b522\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-kube-api-access-4b522\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601489 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601236    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-credential-socket\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601489 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601257    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-istio-token\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601489 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601281    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d059df-4258-48bf-802b-319a753fe612-istio-podinfo\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601622 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601494    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-data\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601622 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601524    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-socket\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601828 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601807    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-credential-socket\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.601888 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.601842    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-certs\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.602091 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.602067    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d059df-4258-48bf-802b-319a753fe612-istiod-ca-cert\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.603342 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.603314    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-envoy\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.603698 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.603682    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d059df-4258-48bf-802b-319a753fe612-istio-podinfo\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.616194 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.616169    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-istio-token\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.616314 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.616287    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b522\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-kube-api-access-4b522\") pod \"e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") " pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.696581 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.696500    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:29.835793 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.835771    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"]
Jul 25 02:06:29.837955 ip-10-0-129-103 kubenswrapper[2562]: W0725 02:06:29.837912    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43d059df_4258_48bf_802b_319a753fe612.slice/crio-bc2281cd0542407cdcc9c7f6c35df14c4154c23eed64d18b32fc54dab44ec32d WatchSource:0}: Error finding container bc2281cd0542407cdcc9c7f6c35df14c4154c23eed64d18b32fc54dab44ec32d: Status 404 returned error can't find the container with id bc2281cd0542407cdcc9c7f6c35df14c4154c23eed64d18b32fc54dab44ec32d
Jul 25 02:06:29.840948 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.840912    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:06:29.841077 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.841022    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:06:29.841077 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:29.841069    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:06:30.112914 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:30.112841    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:30.213908 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:30.213874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:30.544249 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:30.544205    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" event={"ID":"43d059df-4258-48bf-802b-319a753fe612","Type":"ContainerStarted","Data":"1ad9e77b372e7ed0279d9982146f8c5eca2de9a3b160a609204cc3c24eb0340c"}
Jul 25 02:06:30.544454 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:30.544316    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" event={"ID":"43d059df-4258-48bf-802b-319a753fe612","Type":"ContainerStarted","Data":"bc2281cd0542407cdcc9c7f6c35df14c4154c23eed64d18b32fc54dab44ec32d"}
Jul 25 02:06:30.564969 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:30.564920    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" podStartSLOduration=1.564906184 podStartE2EDuration="1.564906184s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-25 02:06:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:06:30.563559081 +0000 UTC m=+3269.034515431" watchObservedRunningTime="2026-07-25 02:06:30.564906184 +0000 UTC m=+3269.035862522"
Jul 25 02:06:30.697223 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:30.697187    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:30.702409 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:30.702385    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:31.549842 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:31.549814    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:31.550850 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:31.550830    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:32.517218 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:32.517190    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:32.615774 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:32.615743    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:33.023909 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:33.023871    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:33.129570 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:33.129540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:33.255658 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:33.255623    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:33.356553 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:33.356469    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:33.461021 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:33.460965    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:33.557575 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:33.557543    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:37.051571 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:37.051540    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:54.671961 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:54.671926    2562 generic.go:356] "Generic (PLEG): container finished" podID="43d059df-4258-48bf-802b-319a753fe612" containerID="1ad9e77b372e7ed0279d9982146f8c5eca2de9a3b160a609204cc3c24eb0340c" exitCode=0
Jul 25 02:06:54.672381 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:54.672011    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" event={"ID":"43d059df-4258-48bf-802b-319a753fe612","Type":"ContainerDied","Data":"1ad9e77b372e7ed0279d9982146f8c5eca2de9a3b160a609204cc3c24eb0340c"}
Jul 25 02:06:54.672434 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:54.672414    2562 scope.go:117] "RemoveContainer" containerID="1ad9e77b372e7ed0279d9982146f8c5eca2de9a3b160a609204cc3c24eb0340c"
Jul 25 02:06:54.673493 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:54.673444    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:06:54.673559 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:54.673508    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:06:54.673559 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:54.673539    2562 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"}
Jul 25 02:06:55.679512 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:55.679471    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" event={"ID":"43d059df-4258-48bf-802b-319a753fe612","Type":"ContainerStarted","Data":"f7923cfcef46905316b65e70e2d7dd842d72e07e1359c59a086227bcba3615a6"}
Jul 25 02:06:55.697299 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:55.697274    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:55.697520 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:55.697493    2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 25 02:06:55.697582 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:55.697565    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 25 02:06:56.697863 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:56.697834    2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 25 02:06:56.698263 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:56.697881    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 25 02:06:57.696890 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:57.696861    2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 25 02:06:57.697074 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:57.696910    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 25 02:06:58.645527 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:58.645494    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:58.697894 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:58.697854    2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 25 02:06:58.698087 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:58.697931    2562 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 25 02:06:58.709217 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:58.709188    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:58.767876 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:58.767842    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:06:59.743701 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:59.743670    2562 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:59.744261 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:59.744225    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:06:59.745175 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:06:59.745156    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:07:03.134726 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:03.134689    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:03.528364 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:03.528224    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:09.340013 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:09.339968    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:09.439897 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:09.439861    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:09.592132 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:09.592051    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"]
Jul 25 02:07:09.592313 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:09.592259    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy" containerID="cri-o://f7923cfcef46905316b65e70e2d7dd842d72e07e1359c59a086227bcba3615a6" gracePeriod=30
Jul 25 02:07:09.612264 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:09.612238    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:09.857823 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:09.857740    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:14.746304 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.746270    2562 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused" start-of-body=
Jul 25 02:07:14.746683 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.746329    2562 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.69:15021/healthz/ready\": dial tcp 10.134.0.69:15021: connect: connection refused"
Jul 25 02:07:14.792228 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.792188    2562 generic.go:356] "Generic (PLEG): container finished" podID="43d059df-4258-48bf-802b-319a753fe612" containerID="f7923cfcef46905316b65e70e2d7dd842d72e07e1359c59a086227bcba3615a6" exitCode=0
Jul 25 02:07:14.792359 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.792262    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" event={"ID":"43d059df-4258-48bf-802b-319a753fe612","Type":"ContainerDied","Data":"f7923cfcef46905316b65e70e2d7dd842d72e07e1359c59a086227bcba3615a6"}
Jul 25 02:07:14.792359 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.792339    2562 scope.go:117] "RemoveContainer" containerID="1ad9e77b372e7ed0279d9982146f8c5eca2de9a3b160a609204cc3c24eb0340c"
Jul 25 02:07:14.858634 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.858583    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:07:14.943335 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943305    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-certs\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.943498 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943346    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-socket\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.943498 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943387    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-credential-socket\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.943498 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943452    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-envoy\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.943656 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943512    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d059df-4258-48bf-802b-319a753fe612-istiod-ca-cert\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.943656 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943542    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-data\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.943656 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943587    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4b522\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-kube-api-access-4b522\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.943656 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943609    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:07:14.943656 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943625    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d059df-4258-48bf-802b-319a753fe612-istio-podinfo\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.943656 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943648    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:07:14.944014 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943671    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-istio-token\") pod \"43d059df-4258-48bf-802b-319a753fe612\" (UID: \"43d059df-4258-48bf-802b-319a753fe612\") "
Jul 25 02:07:14.944014 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943731    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:07:14.944014 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.943928    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43d059df-4258-48bf-802b-319a753fe612-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:07:14.944190 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.944048    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43d059df-4258-48bf-802b-319a753fe612-istiod-ca-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:14.944190 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.944070    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-certs\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:14.944190 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.944083    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-workload-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:14.944190 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.944096    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-credential-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:14.944412 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.944383    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-data" (OuterVolumeSpecName: "istio-data") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:07:14.945936 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.945903    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-kube-api-access-4b522" (OuterVolumeSpecName: "kube-api-access-4b522") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "kube-api-access-4b522". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:07:14.946132 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.946102    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:07:14.946250 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.946171    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43d059df-4258-48bf-802b-319a753fe612-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 02:07:14.946250 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:14.946204    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-istio-token" (OuterVolumeSpecName: "istio-token") pod "43d059df-4258-48bf-802b-319a753fe612" (UID: "43d059df-4258-48bf-802b-319a753fe612"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:07:15.045483 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.045459    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-data\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:15.045483 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.045480    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4b522\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-kube-api-access-4b522\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:15.045673 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.045489    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43d059df-4258-48bf-802b-319a753fe612-istio-podinfo\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:15.045673 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.045498    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43d059df-4258-48bf-802b-319a753fe612-istio-token\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:15.045673 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.045506    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43d059df-4258-48bf-802b-319a753fe612-istio-envoy\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:07:15.798801 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.798770    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"
Jul 25 02:07:15.799260 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.798818    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv" event={"ID":"43d059df-4258-48bf-802b-319a753fe612","Type":"ContainerDied","Data":"bc2281cd0542407cdcc9c7f6c35df14c4154c23eed64d18b32fc54dab44ec32d"}
Jul 25 02:07:15.799260 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.798885    2562 scope.go:117] "RemoveContainer" containerID="f7923cfcef46905316b65e70e2d7dd842d72e07e1359c59a086227bcba3615a6"
Jul 25 02:07:15.819109 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.819085    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"]
Jul 25 02:07:15.823120 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:15.823099    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-a585ca-openshift-default-74dd9dfff8-rgnvv"]
Jul 25 02:07:16.075758 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:16.075685    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43d059df-4258-48bf-802b-319a753fe612" path="/var/lib/kubelet/pods/43d059df-4258-48bf-802b-319a753fe612/volumes"
Jul 25 02:07:29.325945 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:29.325874    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:29.394069 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:29.394034    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:29.448744 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:29.448709    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:33.463314 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:33.463283    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:33.857288 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:33.857217    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:07:38.879334 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:07:38.879298    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:06.348818 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:06.348783    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:11.497166 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:11.497132    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:11.929715 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:11.929680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:17.466402 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:17.466370    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:17.523206 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:17.523174    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:17.751746 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:17.751665    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:18.004816 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:18.004738    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:25.336038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:25.336000    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:28.507253 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:28.507225    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:28.577311 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:28.577274    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:28.628376 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:28.628341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:33.195421 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:33.195388    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:33.619246 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:33.619163    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:39.162692 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:39.162655    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:39.223461 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:39.223428    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:39.416405 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:39.416317    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"]
Jul 25 02:08:39.416618 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:39.416559    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy" containerID="cri-o://a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3" gracePeriod=30
Jul 25 02:08:39.431759 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:39.431726    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:39.671245 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:39.671160    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:44.677061 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.677035    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:08:44.748415 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748323    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-envoy\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.748415 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748380    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z8tbn\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-kube-api-access-z8tbn\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.748415 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748415    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-socket\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.748693 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748435    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-certs\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.748693 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748498    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-token\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.748693 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748522    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-data\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.748693 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748555    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-podinfo\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.748693 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748614    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-credential-socket\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.748693 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748649    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368020b2-dbf6-46c2-a5e7-f28701b59955-istiod-ca-cert\") pod \"368020b2-dbf6-46c2-a5e7-f28701b59955\" (UID: \"368020b2-dbf6-46c2-a5e7-f28701b59955\") "
Jul 25 02:08:44.749039 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748746    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:08:44.749039 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.748804    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:08:44.749168 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.749063    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.749168 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.749085    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-workload-certs\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.749168 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.749087    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:08:44.749320 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.749180    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/368020b2-dbf6-46c2-a5e7-f28701b59955-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:08:44.749872 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.749838    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-data" (OuterVolumeSpecName: "istio-data") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:08:44.750747 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.750712    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-kube-api-access-z8tbn" (OuterVolumeSpecName: "kube-api-access-z8tbn") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "kube-api-access-z8tbn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:08:44.750861 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.750744    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:08:44.750861 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.750781    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 02:08:44.751010 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.750974    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-token" (OuterVolumeSpecName: "istio-token") pod "368020b2-dbf6-46c2-a5e7-f28701b59955" (UID: "368020b2-dbf6-46c2-a5e7-f28701b59955"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:08:44.849578 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.849541    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-token\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.849578 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.849572    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-data\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.849578 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.849580    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-podinfo\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.849578 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.849589    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-credential-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.849861 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.849605    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/368020b2-dbf6-46c2-a5e7-f28701b59955-istiod-ca-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.849861 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.849615    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/368020b2-dbf6-46c2-a5e7-f28701b59955-istio-envoy\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.849861 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.849623    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z8tbn\" (UniqueName: \"kubernetes.io/projected/368020b2-dbf6-46c2-a5e7-f28701b59955-kube-api-access-z8tbn\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:08:44.940376 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:44.940341    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:45.040654 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.040577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:45.237587 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.237552    2562 generic.go:356] "Generic (PLEG): container finished" podID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerID="a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3" exitCode=0
Jul 25 02:08:45.237776 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.237597    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"
Jul 25 02:08:45.237776 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.237630    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" event={"ID":"368020b2-dbf6-46c2-a5e7-f28701b59955","Type":"ContainerDied","Data":"a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3"}
Jul 25 02:08:45.237776 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.237687    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj" event={"ID":"368020b2-dbf6-46c2-a5e7-f28701b59955","Type":"ContainerDied","Data":"18c014c245b08cdc950f9ae85263433b67643728108a52549c6ea3fd2beb542b"}
Jul 25 02:08:45.237776 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.237713    2562 scope.go:117] "RemoveContainer" containerID="a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3"
Jul 25 02:08:45.246778 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.246760    2562 scope.go:117] "RemoveContainer" containerID="6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1"
Jul 25 02:08:45.254642 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.254625    2562 scope.go:117] "RemoveContainer" containerID="a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3"
Jul 25 02:08:45.254905 ip-10-0-129-103 kubenswrapper[2562]: E0725 02:08:45.254887    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3\": container with ID starting with a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3 not found: ID does not exist" containerID="a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3"
Jul 25 02:08:45.254964 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.254914    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3"} err="failed to get container status \"a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3\": rpc error: code = NotFound desc = could not find container \"a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3\": container with ID starting with a5e6302815eecc8ab55774cb1d8e31f2de7b7afeb412448b5837aa53294bb3a3 not found: ID does not exist"
Jul 25 02:08:45.254964 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.254931    2562 scope.go:117] "RemoveContainer" containerID="6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1"
Jul 25 02:08:45.255175 ip-10-0-129-103 kubenswrapper[2562]: E0725 02:08:45.255161    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1\": container with ID starting with 6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1 not found: ID does not exist" containerID="6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1"
Jul 25 02:08:45.255217 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.255178    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1"} err="failed to get container status \"6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1\": rpc error: code = NotFound desc = could not find container \"6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1\": container with ID starting with 6cbd1e7e74c59bfe2cf24280d5397c9336e3e003d45b402ac6f7f24641a401a1 not found: ID does not exist"
Jul 25 02:08:45.259390 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.259366    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"]
Jul 25 02:08:45.261631 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:45.261612    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-d66e66-openshift-default-6f77476bbb-kmhwj"]
Jul 25 02:08:46.076193 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:46.076160    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" path="/var/lib/kubelet/pods/368020b2-dbf6-46c2-a5e7-f28701b59955/volumes"
Jul 25 02:08:50.194128 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:50.194097    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:50.259711 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:50.259680    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:50.312562 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:50.312534    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:54.890621 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:54.890531    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:08:55.320522 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:08:55.320441    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:01.189327 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:01.189291    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:01.288882 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:01.288850    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:01.469528 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:01.469451    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:01.712252 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:01.712216    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:12.379941 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.379907    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:12.402579 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.402545    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"]
Jul 25 02:09:12.402902 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.402869    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d" podUID="82f78176-fe45-4655-b5b0-2e04afc9ea1b" containerName="payload-processing" containerID="cri-o://510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320" gracePeriod=30
Jul 25 02:09:12.479618 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.479577    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:12.532704 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.532673    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:12.691680 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.691656    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 02:09:12.804063 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.804036    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82f78176-fe45-4655-b5b0-2e04afc9ea1b-plugins-config-volume\") pod \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\" (UID: \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\") "
Jul 25 02:09:12.804235 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.804110    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-j9gmg\" (UniqueName: \"kubernetes.io/projected/82f78176-fe45-4655-b5b0-2e04afc9ea1b-kube-api-access-j9gmg\") pod \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\" (UID: \"82f78176-fe45-4655-b5b0-2e04afc9ea1b\") "
Jul 25 02:09:12.804390 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.804367    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82f78176-fe45-4655-b5b0-2e04afc9ea1b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "82f78176-fe45-4655-b5b0-2e04afc9ea1b" (UID: "82f78176-fe45-4655-b5b0-2e04afc9ea1b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:09:12.806140 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.806118    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82f78176-fe45-4655-b5b0-2e04afc9ea1b-kube-api-access-j9gmg" (OuterVolumeSpecName: "kube-api-access-j9gmg") pod "82f78176-fe45-4655-b5b0-2e04afc9ea1b" (UID: "82f78176-fe45-4655-b5b0-2e04afc9ea1b"). InnerVolumeSpecName "kube-api-access-j9gmg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:09:12.905109 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.905076    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-j9gmg\" (UniqueName: \"kubernetes.io/projected/82f78176-fe45-4655-b5b0-2e04afc9ea1b-kube-api-access-j9gmg\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:12.905109 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:12.905103    2562 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/82f78176-fe45-4655-b5b0-2e04afc9ea1b-plugins-config-volume\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:13.376784 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.376751    2562 generic.go:356] "Generic (PLEG): container finished" podID="82f78176-fe45-4655-b5b0-2e04afc9ea1b" containerID="510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320" exitCode=0
Jul 25 02:09:13.377027 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.376786    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d" event={"ID":"82f78176-fe45-4655-b5b0-2e04afc9ea1b","Type":"ContainerDied","Data":"510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320"}
Jul 25 02:09:13.377027 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.376806    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"
Jul 25 02:09:13.377027 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.376847    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d" event={"ID":"82f78176-fe45-4655-b5b0-2e04afc9ea1b","Type":"ContainerDied","Data":"02f550d7f4b6fb8ebd0b647b3338d3d061aea34d66dfbfcad7bbd4d6cd3541a2"}
Jul 25 02:09:13.377027 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.376871    2562 scope.go:117] "RemoveContainer" containerID="510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320"
Jul 25 02:09:13.386163 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.386147    2562 scope.go:117] "RemoveContainer" containerID="510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320"
Jul 25 02:09:13.386430 ip-10-0-129-103 kubenswrapper[2562]: E0725 02:09:13.386407    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320\": container with ID starting with 510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320 not found: ID does not exist" containerID="510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320"
Jul 25 02:09:13.386471 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.386433    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320"} err="failed to get container status \"510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320\": rpc error: code = NotFound desc = could not find container \"510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320\": container with ID starting with 510c270e30840173499bcf5dd53cb973171cef4ff51091ce5c7d7845b3879320 not found: ID does not exist"
Jul 25 02:09:13.397545 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.397518    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"]
Jul 25 02:09:13.399305 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:13.399286    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-ab0ae6-579745dff8-6sk7d"]
Jul 25 02:09:14.075807 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:14.075765    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82f78176-fe45-4655-b5b0-2e04afc9ea1b" path="/var/lib/kubelet/pods/82f78176-fe45-4655-b5b0-2e04afc9ea1b/volumes"
Jul 25 02:09:16.906338 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:16.906307    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:17.319335 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:17.319243    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:23.126399 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:23.126365    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:23.194309 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:23.194275    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:23.376361 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:23.376330    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"]
Jul 25 02:09:23.377312 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:23.377206    2562 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy" containerID="cri-o://6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87" gracePeriod=30
Jul 25 02:09:23.401050 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:23.401026    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:23.661898 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:23.661821    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:28.643490 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.643470    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 02:09:28.758089 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758024    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istiod-ca-cert\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758089 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758058    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-credential-socket\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758089 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758078    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-envoy\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758343 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758094    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-certs\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758343 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758155    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-token\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758343 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758198    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-podinfo\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758343 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758225    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-data\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758343 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758284    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-socket\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758343 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758305    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:09:28.758652 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758313    2562 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dksb\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-kube-api-access-8dksb\") pod \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\" (UID: \"8af909ce-3356-47c4-8e35-2a9ad7e3a194\") "
Jul 25 02:09:28.758652 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758536    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Jul 25 02:09:28.758757 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758707    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:09:28.758757 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758727    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:09:28.758861 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758781    2562 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istiod-ca-cert\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:28.758861 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.758801    2562 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-credential-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:28.759229 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.759201    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-data" (OuterVolumeSpecName: "istio-data") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:09:28.760426 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.760406    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-kube-api-access-8dksb" (OuterVolumeSpecName: "kube-api-access-8dksb") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "kube-api-access-8dksb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:09:28.760516 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.760452    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-token" (OuterVolumeSpecName: "istio-token") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Jul 25 02:09:28.760644 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.760625    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Jul 25 02:09:28.760711 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.760625    2562 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "8af909ce-3356-47c4-8e35-2a9ad7e3a194" (UID: "8af909ce-3356-47c4-8e35-2a9ad7e3a194"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Jul 25 02:09:28.859965 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.859938    2562 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-token\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:28.859965 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.859960    2562 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-podinfo\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:28.860135 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.859974    2562 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-data\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:28.860135 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.860009    2562 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-socket\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:28.860135 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.860022    2562 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dksb\" (UniqueName: \"kubernetes.io/projected/8af909ce-3356-47c4-8e35-2a9ad7e3a194-kube-api-access-8dksb\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:28.860135 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.860035    2562 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-istio-envoy\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:28.860135 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:28.860046    2562 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8af909ce-3356-47c4-8e35-2a9ad7e3a194-workload-certs\") on node \"ip-10-0-129-103.ec2.internal\" DevicePath \"\""
Jul 25 02:09:29.459033 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.459001    2562 generic.go:356] "Generic (PLEG): container finished" podID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerID="6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87" exitCode=0
Jul 25 02:09:29.459196 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.459042    2562 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"
Jul 25 02:09:29.459196 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.459081    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" event={"ID":"8af909ce-3356-47c4-8e35-2a9ad7e3a194","Type":"ContainerDied","Data":"6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87"}
Jul 25 02:09:29.459196 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.459133    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29" event={"ID":"8af909ce-3356-47c4-8e35-2a9ad7e3a194","Type":"ContainerDied","Data":"3a0faf1e615a14124266e55b01faa1a939026c5f211eee16848e189ce22d2e4f"}
Jul 25 02:09:29.459196 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.459157    2562 scope.go:117] "RemoveContainer" containerID="6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87"
Jul 25 02:09:29.468397 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.468382    2562 scope.go:117] "RemoveContainer" containerID="a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032"
Jul 25 02:09:29.475958 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.475942    2562 scope.go:117] "RemoveContainer" containerID="6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87"
Jul 25 02:09:29.476228 ip-10-0-129-103 kubenswrapper[2562]: E0725 02:09:29.476207    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87\": container with ID starting with 6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87 not found: ID does not exist" containerID="6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87"
Jul 25 02:09:29.476281 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.476244    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87"} err="failed to get container status \"6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87\": rpc error: code = NotFound desc = could not find container \"6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87\": container with ID starting with 6db5c80f0a9e588c50afb4fc9c967e505f5cc26b0b957b11bd97572201827e87 not found: ID does not exist"
Jul 25 02:09:29.476281 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.476262    2562 scope.go:117] "RemoveContainer" containerID="a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032"
Jul 25 02:09:29.476495 ip-10-0-129-103 kubenswrapper[2562]: E0725 02:09:29.476480    2562 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032\": container with ID starting with a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032 not found: ID does not exist" containerID="a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032"
Jul 25 02:09:29.476542 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.476498    2562 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032"} err="failed to get container status \"a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032\": rpc error: code = NotFound desc = could not find container \"a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032\": container with ID starting with a960d46946028f2f566917fc1bd7ba6551705799a6dade6eee84bce9266d8032 not found: ID does not exist"
Jul 25 02:09:29.480439 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.480419    2562 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"]
Jul 25 02:09:29.484540 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.484520    2562 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-ab0ae6-openshift-default-59fc8c788-trs29"]
Jul 25 02:09:29.771204 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.771145    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9ftn5_83ad3d9b-ebc6-4ba0-8469-4ceb60152f68/manager/0.log"
Jul 25 02:09:29.890303 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:29.890279    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jc4xw_7a6ddf44-2336-4d83-8fdc-fedbb6a9a286/manager/0.log"
Jul 25 02:09:30.017020 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:30.016996    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-859cc5f8b9-n48bk_d05574de-837a-42b0-84da-65f4210f7d12/manager/0.log"
Jul 25 02:09:30.079936 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:30.079873    2562 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" path="/var/lib/kubelet/pods/8af909ce-3356-47c4-8e35-2a9ad7e3a194/volumes"
Jul 25 02:09:30.138385 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:30.138366    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nzz8d_ba9d170c-06a0-4247-976f-611bcbe95ff3/server/0.log"
Jul 25 02:09:30.269692 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:30.269669    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pmv72_dca5076c-ffa3-4fd0-a8cb-227a91400d8f/manager/0.log"
Jul 25 02:09:30.392688 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:30.392669    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7587ff6c45-m2xxg_bf113b42-7ff9-4159-89d4-429204ee9bf4/copy-manifests/0.log"
Jul 25 02:09:30.404493 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:30.404471    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7587ff6c45-m2xxg_bf113b42-7ff9-4159-89d4-429204ee9bf4/manager/0.log"
Jul 25 02:09:32.972845 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:32.972817    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fvgq4_0381d3ae-a5f4-4465-a57e-d97a2300f0bc/manager/0.log"
Jul 25 02:09:33.088610 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:33.088589    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-j2q5r_68f1e9de-a0d2-4eb3-8d48-9bada87fa201/manager/0.log"
Jul 25 02:09:33.322597 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:33.322536    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hlxgt_59b701d4-aa2a-43c7-b164-baf5ff77989f/registry-server/0.log"
Jul 25 02:09:33.439689 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:33.439665    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-scbkv_bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb/manager/0.log"
Jul 25 02:09:33.674142 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:33.674119    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-m4qb8_d1dea10f-e012-4dc5-b06c-df4d4a1ccb86/manager/0.log"
Jul 25 02:09:34.041211 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:34.041142    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg_1fd0785d-3ce7-469d-9671-bda9d2fd9603/istio-proxy/0.log"
Jul 25 02:09:34.398772 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:34.398745    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-789f49c94d-rq8fr_ab660b12-c049-4d2d-a814-8247e818ebc2/kube-auth-proxy/0.log"
Jul 25 02:09:34.517598 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:34.517562    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5jk8x_7b75df48-087c-4718-a246-d51b95fd1669/istio-proxy/1.log"
Jul 25 02:09:34.643846 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:34.643823    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vw2r7_bec36db6-f077-4467-ba86-86b131ad2f4d/payload-pre-processing/0.log"
Jul 25 02:09:34.768504 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:34.768434    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-j8tq2_48ed41f2-58a1-4f84-9c9c-82525a777da2/payload-processing/0.log"
Jul 25 02:09:35.248406 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:35.248382    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-qmvvv_d8ea409a-261f-40af-a816-e179423df4b2/main/0.log"
Jul 25 02:09:35.598273 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:35.598200    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68fvnb5q_9cca9bcf-c1dc-4dd1-927b-762d26f7898f/main/0.log"
Jul 25 02:09:35.716460 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:35.716436    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-ds5d2_7eda39df-2fc7-4126-92ef-851c6431cf38/main/0.log"
Jul 25 02:09:41.266804 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:41.266769    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:41.364174 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:41.364133    2562 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log" limit=1024
Jul 25 02:09:43.923958 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:43.923929    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-c46qr_e1fb37b0-fa56-4167-873a-c72bb0947ebd/global-pull-secret-syncer/0.log"
Jul 25 02:09:44.046062 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:44.046036    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8qxdx_b8237157-cb64-498e-8759-7715d46af050/konnectivity-agent/0.log"
Jul 25 02:09:44.134445 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:44.134418    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-129-103.ec2.internal_1ed3ed7be1c2695b0c97f92390b9feef/haproxy/0.log"
Jul 25 02:09:49.087775 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:49.087728    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-fvgq4_0381d3ae-a5f4-4465-a57e-d97a2300f0bc/manager/0.log"
Jul 25 02:09:49.121813 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:49.121789    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-j2q5r_68f1e9de-a0d2-4eb3-8d48-9bada87fa201/manager/0.log"
Jul 25 02:09:49.198541 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:49.198516    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hlxgt_59b701d4-aa2a-43c7-b164-baf5ff77989f/registry-server/0.log"
Jul 25 02:09:49.379952 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:49.379926    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-scbkv_bfb55f24-5f14-4b8c-b3b6-0aa2b7f785bb/manager/0.log"
Jul 25 02:09:49.485170 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:49.485143    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-m4qb8_d1dea10f-e012-4dc5-b06c-df4d4a1ccb86/manager/0.log"
Jul 25 02:09:51.342096 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.342064    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-68b7fc9574-j2wx8_431a96c2-8ba7-4163-9b06-cf3e14872286/monitoring-plugin/0.log"
Jul 25 02:09:51.466549 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.466485    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x65db_493c913a-3a85-4898-9045-967a926db33a/node-exporter/0.log"
Jul 25 02:09:51.488243 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.488216    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x65db_493c913a-3a85-4898-9045-967a926db33a/kube-rbac-proxy/0.log"
Jul 25 02:09:51.514928 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.514907    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-x65db_493c913a-3a85-4898-9045-967a926db33a/init-textfile/0.log"
Jul 25 02:09:51.734320 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.734227    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4dc8a3d3-576d-4418-8956-0a1496f83485/prometheus/0.log"
Jul 25 02:09:51.759038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.759019    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4dc8a3d3-576d-4418-8956-0a1496f83485/config-reloader/0.log"
Jul 25 02:09:51.783151 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.783126    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4dc8a3d3-576d-4418-8956-0a1496f83485/thanos-sidecar/0.log"
Jul 25 02:09:51.802899 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.802885    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4dc8a3d3-576d-4418-8956-0a1496f83485/kube-rbac-proxy-web/0.log"
Jul 25 02:09:51.829881 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.829859    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4dc8a3d3-576d-4418-8956-0a1496f83485/kube-rbac-proxy/0.log"
Jul 25 02:09:51.859201 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.859182    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4dc8a3d3-576d-4418-8956-0a1496f83485/kube-rbac-proxy-thanos/0.log"
Jul 25 02:09:51.882297 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:51.882278    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_4dc8a3d3-576d-4418-8956-0a1496f83485/init-config-reloader/0.log"
Jul 25 02:09:54.251558 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251528    2562 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"]
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251875    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251886    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251895    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251901    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251912    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f78176-fe45-4655-b5b0-2e04afc9ea1b" containerName="payload-processing"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251918    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f78176-fe45-4655-b5b0-2e04afc9ea1b" containerName="payload-processing"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251924    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251930    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251944    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251950    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251959    2562 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.251964    2562 state_mem.go:107] "Deleted CPUSet assignment" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.252040    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="43d059df-4258-48bf-802b-319a753fe612" containerName="istio-proxy"
Jul 25 02:09:54.252038 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.252047    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="8af909ce-3356-47c4-8e35-2a9ad7e3a194" containerName="istio-proxy"
Jul 25 02:09:54.252540 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.252057    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f78176-fe45-4655-b5b0-2e04afc9ea1b" containerName="payload-processing"
Jul 25 02:09:54.252540 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.252065    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy"
Jul 25 02:09:54.252540 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.252073    2562 memory_manager.go:368] "RemoveStaleState removing state" podUID="368020b2-dbf6-46c2-a5e7-f28701b59955" containerName="istio-proxy"
Jul 25 02:09:54.255058 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.254973    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.258928 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.258910    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qpj2w\"/\"kube-root-ca.crt\""
Jul 25 02:09:54.259088 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.259071    2562 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-qpj2w\"/\"openshift-service-ca.crt\""
Jul 25 02:09:54.259797 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.259779    2562 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-qpj2w\"/\"default-dockercfg-b5m6f\""
Jul 25 02:09:54.265436 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.265417    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"]
Jul 25 02:09:54.363447 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.363424    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-podres\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.363542 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.363463    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-lib-modules\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.363542 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.363513    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-proc\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.363542 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.363539    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-sys\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.363650 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.363598    2562 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6spg9\" (UniqueName: \"kubernetes.io/projected/65398e9a-cd01-4829-a884-32259b525e5c-kube-api-access-6spg9\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.464830 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.464807    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-podres\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.464932 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.464846    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-lib-modules\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.464932 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.464884    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-proc\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.464932 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.464918    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-sys\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.465122 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.464952    2562 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6spg9\" (UniqueName: \"kubernetes.io/projected/65398e9a-cd01-4829-a884-32259b525e5c-kube-api-access-6spg9\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.465122 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.464999    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-lib-modules\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.465122 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.465009    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-podres\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.465122 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.465049    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-proc\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.465122 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.465057    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/65398e9a-cd01-4829-a884-32259b525e5c-sys\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.472600 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.472582    2562 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6spg9\" (UniqueName: \"kubernetes.io/projected/65398e9a-cd01-4829-a884-32259b525e5c-kube-api-access-6spg9\") pod \"perf-node-gather-daemonset-5hk94\" (UID: \"65398e9a-cd01-4829-a884-32259b525e5c\") " pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.566218 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.566167    2562 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:54.697747 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:54.697715    2562 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"]
Jul 25 02:09:54.700482 ip-10-0-129-103 kubenswrapper[2562]: W0725 02:09:54.700456    2562 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod65398e9a_cd01_4829_a884_32259b525e5c.slice/crio-c4e82a0b9859c736930f42affba261968108eee3f3199c9f98a6951a7f9d428f WatchSource:0}: Error finding container c4e82a0b9859c736930f42affba261968108eee3f3199c9f98a6951a7f9d428f: Status 404 returned error can't find the container with id c4e82a0b9859c736930f42affba261968108eee3f3199c9f98a6951a7f9d428f
Jul 25 02:09:55.587637 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:55.587602    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94" event={"ID":"65398e9a-cd01-4829-a884-32259b525e5c","Type":"ContainerStarted","Data":"3dcd775a08465714dc547472f1e855e72723b0d2b1d083416394da422259f2fa"}
Jul 25 02:09:55.588171 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:55.587663    2562 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94" event={"ID":"65398e9a-cd01-4829-a884-32259b525e5c","Type":"ContainerStarted","Data":"c4e82a0b9859c736930f42affba261968108eee3f3199c9f98a6951a7f9d428f"}
Jul 25 02:09:55.588171 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:55.587690    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:09:55.603640 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:55.603590    2562 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94" podStartSLOduration=1.6035772590000001 podStartE2EDuration="1.603577259s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-25 02:09:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-25 02:09:55.601991163 +0000 UTC m=+3474.072947491" watchObservedRunningTime="2026-07-25 02:09:55.603577259 +0000 UTC m=+3474.074533652"
Jul 25 02:09:56.368167 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:56.368135    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b6rzf_bb972c6d-e9ed-4b05-8817-1bed95eaae32/dns/0.log"
Jul 25 02:09:56.386349 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:56.386324    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-b6rzf_bb972c6d-e9ed-4b05-8817-1bed95eaae32/kube-rbac-proxy/0.log"
Jul 25 02:09:56.450237 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:56.450207    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hljng_2f15f82d-6ba2-4275-a812-d1cd89ea9f69/dns-node-resolver/0.log"
Jul 25 02:09:57.038299 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:57.038266    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-szrr5_52431c6d-7b58-4ecb-b1d1-d39ad37f1fd8/node-ca/0.log"
Jul 25 02:09:57.940854 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:57.940821    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg_1fd0785d-3ce7-469d-9671-bda9d2fd9603/istio-proxy/0.log"
Jul 25 02:09:58.129282 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:58.129253    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-789f49c94d-rq8fr_ab660b12-c049-4d2d-a814-8247e818ebc2/kube-auth-proxy/0.log"
Jul 25 02:09:58.157663 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:58.157638    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5jk8x_7b75df48-087c-4718-a246-d51b95fd1669/istio-proxy/0.log"
Jul 25 02:09:58.183851 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:58.183813    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5jk8x_7b75df48-087c-4718-a246-d51b95fd1669/istio-proxy/1.log"
Jul 25 02:09:58.213029 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:58.212931    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vw2r7_bec36db6-f077-4467-ba86-86b131ad2f4d/payload-pre-processing/0.log"
Jul 25 02:09:58.239609 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:58.239585    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-j8tq2_48ed41f2-58a1-4f84-9c9c-82525a777da2/payload-processing/0.log"
Jul 25 02:09:59.063177 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:59.063132    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-svbfp_5bd143fd-07cf-4981-981c-7f515ac8336f/serve-healthcheck-canary/0.log"
Jul 25 02:09:59.675456 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:59.675415    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r9s4m_c5577022-409f-4518-a9bd-77f7b0ffbcf0/kube-rbac-proxy/0.log"
Jul 25 02:09:59.693520 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:59.693498    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r9s4m_c5577022-409f-4518-a9bd-77f7b0ffbcf0/exporter/0.log"
Jul 25 02:09:59.714063 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:09:59.714044    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-r9s4m_c5577022-409f-4518-a9bd-77f7b0ffbcf0/extractor/0.log"
Jul 25 02:10:01.602568 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:01.602541    2562 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-qpj2w/perf-node-gather-daemonset-5hk94"
Jul 25 02:10:01.759972 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:01.759930    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-9ftn5_83ad3d9b-ebc6-4ba0-8469-4ceb60152f68/manager/0.log"
Jul 25 02:10:01.966729 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:01.966701    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-jc4xw_7a6ddf44-2336-4d83-8fdc-fedbb6a9a286/manager/0.log"
Jul 25 02:10:02.123351 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:02.123321    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-859cc5f8b9-n48bk_d05574de-837a-42b0-84da-65f4210f7d12/manager/0.log"
Jul 25 02:10:02.143155 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:02.143129    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-nzz8d_ba9d170c-06a0-4247-976f-611bcbe95ff3/server/0.log"
Jul 25 02:10:02.214578 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:02.214555    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-pmv72_dca5076c-ffa3-4fd0-a8cb-227a91400d8f/manager/0.log"
Jul 25 02:10:02.292128 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:02.292071    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7587ff6c45-m2xxg_bf113b42-7ff9-4159-89d4-429204ee9bf4/manager/0.log"
Jul 25 02:10:02.311119 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:02.311099    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7587ff6c45-m2xxg_bf113b42-7ff9-4159-89d4-429204ee9bf4/copy-manifests/0.log"
Jul 25 02:10:04.593431 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:04.593402    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg_1fd0785d-3ce7-469d-9671-bda9d2fd9603/istio-proxy/0.log"
Jul 25 02:10:04.756373 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:04.756345    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-789f49c94d-rq8fr_ab660b12-c049-4d2d-a814-8247e818ebc2/kube-auth-proxy/0.log"
Jul 25 02:10:04.774858 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:04.774818    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5jk8x_7b75df48-087c-4718-a246-d51b95fd1669/istio-proxy/0.log"
Jul 25 02:10:04.800119 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:04.800093    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5jk8x_7b75df48-087c-4718-a246-d51b95fd1669/istio-proxy/1.log"
Jul 25 02:10:04.822945 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:04.822911    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vw2r7_bec36db6-f077-4467-ba86-86b131ad2f4d/payload-pre-processing/0.log"
Jul 25 02:10:04.845402 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:04.845302    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-j8tq2_48ed41f2-58a1-4f84-9c9c-82525a777da2/payload-processing/0.log"
Jul 25 02:10:06.275320 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:06.275244    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-t58pg_1fd0785d-3ce7-469d-9671-bda9d2fd9603/istio-proxy/0.log"
Jul 25 02:10:06.441323 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:06.441290    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-789f49c94d-rq8fr_ab660b12-c049-4d2d-a814-8247e818ebc2/kube-auth-proxy/0.log"
Jul 25 02:10:06.464357 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:06.464330    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5jk8x_7b75df48-087c-4718-a246-d51b95fd1669/istio-proxy/0.log"
Jul 25 02:10:06.489263 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:06.489232    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-5jk8x_7b75df48-087c-4718-a246-d51b95fd1669/istio-proxy/1.log"
Jul 25 02:10:06.507678 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:06.507653    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-687dbb6cf6-vw2r7_bec36db6-f077-4467-ba86-86b131ad2f4d/payload-pre-processing/0.log"
Jul 25 02:10:06.527160 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:06.527098    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5cc6cd444b-j8tq2_48ed41f2-58a1-4f84-9c9c-82525a777da2/payload-processing/0.log"
Jul 25 02:10:08.879583 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:08.879549    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fqvkq_b3144e4f-a7b6-454c-bec6-e90aa86bbce3/migrator/0.log"
Jul 25 02:10:08.902579 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:08.902549    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-fqvkq_b3144e4f-a7b6-454c-bec6-e90aa86bbce3/graceful-termination/0.log"
Jul 25 02:10:10.461076 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.461047    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vl7nz_dd88ee59-a0f7-4608-8c0a-77cc79d6e25b/kube-multus-additional-cni-plugins/0.log"
Jul 25 02:10:10.478996 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.478950    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vl7nz_dd88ee59-a0f7-4608-8c0a-77cc79d6e25b/egress-router-binary-copy/0.log"
Jul 25 02:10:10.498045 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.498024    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vl7nz_dd88ee59-a0f7-4608-8c0a-77cc79d6e25b/cni-plugins/0.log"
Jul 25 02:10:10.516663 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.516644    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vl7nz_dd88ee59-a0f7-4608-8c0a-77cc79d6e25b/bond-cni-plugin/0.log"
Jul 25 02:10:10.534586 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.534568    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vl7nz_dd88ee59-a0f7-4608-8c0a-77cc79d6e25b/routeoverride-cni/0.log"
Jul 25 02:10:10.552602 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.552585    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vl7nz_dd88ee59-a0f7-4608-8c0a-77cc79d6e25b/whereabouts-cni-bincopy/0.log"
Jul 25 02:10:10.571418 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.571396    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vl7nz_dd88ee59-a0f7-4608-8c0a-77cc79d6e25b/whereabouts-cni/0.log"
Jul 25 02:10:10.672150 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.672128    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wwkc9_95e94611-4840-4771-ac29-e1598ac1a3f4/kube-multus/0.log"
Jul 25 02:10:10.773604 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.773539    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w6nf6_725501d7-58c0-4b02-9e3f-8e931b0d0dfc/network-metrics-daemon/0.log"
Jul 25 02:10:10.791427 ip-10-0-129-103 kubenswrapper[2562]: I0725 02:10:10.791411    2562 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-w6nf6_725501d7-58c0-4b02-9e3f-8e931b0d0dfc/kube-rbac-proxy/0.log"